Maine Library of Geographic Information Board Meeting
Date: Wednesday, September 20, 2006
Time: 10:00 a.m. – 12:30 p.m.
Place: Cross Office Building – Conf Room 105
- Approval of the August 2 meeting minutes
- Update on GeoLibrary Board and GIS Executive Council action items – Jim
- GeoArchives Project – Jim Henderson, State Archivist
- Discussion on the County Report – Michael Terner, Applied Geographics, Inc.
- Board Membership – Jim Page
- Election of Board Chair and Co-Chair
- Subcommittee Reports/Appointments
- Financial – Jim & Dan
- Policy & Marketing – Marilyn
- Technical – Will
- Status on the GEOPortal – Will
- Guidelines for Portal Administrator – Marilyn
- Orthoimagery Project – Larry
- Parcel Grants - Larry
- Plan to purchase intermediate products– Dan
Special Meeting: Discussion/review of the MOU between the GeoLibrary Board and the GIS Executive Council
Future Meeting Agenda Items
- Legislative Planning
- USGS Initiative
NEXT SCHEDULED MEETING: Wednesday October 18, 2006 10:00 a.m. – 12:30 p.m., Burton M. Cross Building, Conference Room 105.
Jim Page, Chair (representing vendors)
Marilyn Lutz, Co-Chair (representing the Univ. of Maine System)
Nancy Armentrout, Designee for the Commissioner of DAFS
William Hanson, Representing Real Estate & Development Interests
Barbara Charry, Representing Environmental Interests
Christopher Kroot, Representing State GIS Functions
David Blocher, Designee for the Chief Information Officer
Will Mitchell, Representing vendors (by conference call)
Ken Murchison, Representing Regional Councils (by conference call)
Jim Ward, Representing Municipal Government
Ken Honey, Representing Statewide Association of Counties
Staff to the Board
Ray Halperin, former Board member
Jim Henderson, State Archivist
Michael Terner, Applied Geographics, Inc.
Rich Sutton, Applied Geographics, Inc (by conference call)
Don Garrold, Town of Searsport
Agenda Item #1: Approval of the August 16th meeting minutes
After giving Board members a few minutes to review the minutes, the Chair entertained a motion to approve. Marilyn Lutz moved to approve the minutes as written and Christopher Kroot seconded. The Board voted unanimously to approve the minutes as written.
Agenda Item #2: Update on the GeoLibrary Board and GIS Executive Council action items
The chair noted that a special meeting to discuss a Memorandum of Agreement with the GIS Executive Council was scheduled after the Board meeting. Further discussion was differed to this meeting.
Agenda Item #3: GeoArchives Project – Jim Henderson, State Archivist
Jim Henderson described the GeoArchive project as a successful test case for the archiving of GIS data using the GeoLibrary as a model. The GeoLibrary Board had appropriated $7,700 towards this project which also had funding from the National Historical Publications and Records Commission (NHPRC). One data layer, the Maine minor civil divisions, was selected as a sample data set and MEGIS staff provided assistance. The project has worked well and has been quite a learning experience for Archives.
Basically, this project targeted GIS data layers already in the GeoLibrary as a controlled standardized test universe. It focused exclusively on the data layers and specifically excluded records of publications created from that data, such as digital or printed maps. Those will be scheduled with records of the agencies producing those records. The strategy focused on using the proprietary production and access system used by the State of Maine (ESRI® GIS software and Oracle® databases) to integrate archival data layers into the system that manages critical GIS data: the road networks, political boundaries, hydrography, etc. As these systems are upgraded, migrated, or replaced, the archival data layers will move with the current data layers.
A complete copy of the presentation can be viewed as addendum # 1 to the minutes.
In the discussion that followed, kudos were given to the project all around and several questions were raised. 1) How would this be paid for in the future? The best answer seems to be as an enterprise service paid for by all users. 2) What will be the future role of ArcInfo 9.2 which is said to have a built in archival functionality? Christopher Kroot gave a brief explanation of the 9.2 beta being run at DEP. The summation was that testing after release was the only way to know if 9.2 could be used out of the box. 3) What were the obstacles? The biggest was simple communications and procedures. There were also technical issues with XML and determining when a data layer had been updated. 4) Was non-ESRI data considered? Not in this initial project but that is certainly a future consideration. It was also suggested that such a successful project for minimal funding presented a good marketing opportunity for the GeoLibrary.
Agenda Item #4: Discussion on the County Report – Michael Terner, Applied Geographics, Inc.
The final report for a GIS Needs Assessment & Requirements Analysis for Maine County Government was issued June 2006. This study provides a requirements analysis and implementation framework for enhancing geospatial technology activities beneficial to Maine County government. The report notes that Maine should seek to develop geographic information system (GIS) capabilities to promote current and future County administrative and planning initiatives and integrate these more effectively with the State’s evolving geospatial services and data resources. In pursuit of these high level objectives, Maine should look to:
- Increase County utilization of geospatial data and technical tools in areas where GIS provides benefits through increased efficiency, cost avoidance, or service enhancement,
- Establish and strengthen regional service centers for providing geospatial technology access in such a way that County needs are addressed, and
- Identify and secure funding mechanisms that will provide baseline support for County geospatial needs in a recurring fashion.
This study provides a strategy to address these objectives in order to move the counties, the state and other participating stakeholders toward more effective and integrated utilization of geospatial technology.
Michael Terner from Applied Geographics, Inc. gave the following presentation to the Maine GeoLibrary Board as a result of this study/final report.
A complete copy of the presentation can be viewed as addendum # 2 to the minutes.
Funding is one of the key obstacles and it is hard to get the counties to work with other agencies. However, counties seem to be in agreement they would like this kind of capability. This would be a new reality getting the counties working with the state.
It was noted this was an excellent report and what other audience should we share this information with: MMA and SPO
It was recommended the Board review objectives within the report; set priorities, and bring in allies. The Board does need to perform a sorting process.
Among the items discussed and that need to be considered are:Who are some of the leaders to foster change? The study/report did identify a lot of the good work that is going on within some of the counties. The Board needs to consider: the state’s interest; how counties can help get the activity going; benefits to the state having universal parcel coverage; identifying the benefits for all. Have counties endorsed what that state has been doing with parcels and have they incorporated into their GIS? The sense of the counties is they would like to see this happen but don’t know how they can do this themselves. Have we explored relationships between counties and Regional Planning Councils? Not a peaceable kingdom; need to divide roles to avoid competition; need formalized rules not eating from the same plate. Within different regions of the state different scenarios could play out. The Board needs to revisit its limitations from the Board’s point of view. Board cannot fund positions but we can buy them a computer. There is lots of “food for thought” in this material. This was a small project with some real meat to itThe Board may want to think about a special agenda item – what action items will the Board develop – merits a lot more discussion.
Do we want to get in between the municipalities and counties? We are ideally setup to do that; we can provide some good ideas. Can we identify the people we can enable? We can give some influence; we can help out; somebody there has to champion this.
It is important to think about having a GIS area that is functional. The Board could provide some education to available opportunities.
- Post final copy of the report on the GeoLibary Board website
- Send E-mail to participants when final report is posted for input and pick up some better names
- Dan continues to work with folks within the various counties
- Share report with other entities: MMA, SPO, CPAC,
Agenda Item #5: Board Membership – Jim Page, Chair
The Board finally appears to be in good shape for filling vacant seats. Thank you to all the folks who stayed involved to get this process moving forward.
- Jim Ward, GIS Coordinator/Manager for the City of Lewiston has been appointed to represent municipalities
- Gary Duplisea has been nominated to represent public utilities
- Ken Honey has been appointed to represent counties
- Nancy Armentrout has been appointed as the Designee for the Commissioner of DAFS
- Christopher Kroot has been appointed to represent state GIS functions
- Dan Coker will be nominated to represent environmental interests
Everyone was asked to introduce themselves and give a brief summary of their expertise.
Agenda Item #6: Election of Board Chair and Co-Chair
The role of the Chair/Co-Chair is to run the Board meeting for one year, organize standing committees and appoint Chairs for standing committees. Currently there are three standing committees: Technical Subcommittee (Will Mitchell, Chair); Policy, Planning and Marketing Subcommittee (Marilyn Lutz, Chair); and Financial Subcommittee (Jim Page, Chair).
One of the goals for the Board moving forward is to secure additional funding via bonds. We need to put together a very well crafted/organized message to bring to the Legislature. Driving Board initiatives forward requires funding and support for implementation. It was noted that having someone from outside state agencies as Chair is a positive for the Board.
Jim and Marilyn agreed to serve for one more year as Chair and Co-Chair if the Board so desired. With new members coming on board and being the largest change in Board membership ever experienced, it was recommended to retain the current Chair and Co-Chair for another 6 months. Christopher Kroot nominated the current Chair and Co-Chair for another year. Motion to cease nominations was made, seconded and approved unanimously. The Board members voted by secret ballot; members left the room and staff took votes from the two members conferencing in via telephone.
After staff counting the secret ballots and 2 votes via conference phone, the current Chair and Co-Chair was unanimously (11 Yes votes) re-elected to serve one more year.
A discussion followed about the three current standing committees. Are there any changes that need or should be made in the structure of the committees? It was noted there is a need to create some significant marketing documents.
It was mentioned, in light of the Memorandum of Understanding (MOU) between the Maine GeoLibrary Board and the GIS Executive Council, the MOU needs to be established, sanctioned and it has to happen. It needs to be formalized and signed by both the Board Chair (Jim Page) and the Council Chair (Christopher Kroot).
A question was raised regarding the current members of the current standing committees and their willingness to continue working on these standing committees. David Blocher volunteered to serve on the Policy, Planning and Marketing Subcommittee. It was suggested to set up a Governmental Affairs Subcommittee.
The Chair asked that Board members volunteer to serve on at least one subcommittee and meet at least once a month.
Action Item: Please e-mail Jim Page with your choice of the subcommittee you would like to serve on.
Marilyn talked about the scope of the Policy, Planning and Marketing subcommittee and the need for preparing marketing documents for the Legislature. This subcommittee drafts language for needed policies and bring before the Board for review/comment and adoption.
The Technical subcommittee works on technical proposals/projects and recommendations are brought to the Board for review/comments and approval. They serve as an oversight project management for tasks taken on by the Board. This is a very important subcommittee and very active.
The Board would also highly benefit from having a government liaison to work with the Governor’s Office, State Planning Office and the Legislature.
Agenda Item #7: Subcommittee Reports
Financial Subcommittee – Jim Page, Chair
Jim reported on the status of Board finances from a spreadsheet put together by Dan Walters. He noted that the Board should be very careful on how the remaining funds are spent.
Policy, Planning and Marketing Subcommittee – Marilyn Lutz, Chair
Marilyn reported the last action item from this subcommittee was the drafted language of the data policy reviewed by the Board and adopted. Other work included the MOU.
Technical Subcommittee – Will Mitchell, Chair
Will reported there has been a lot of communication with the vendor regarding the Geo Portal.
- They have been working on issues with functionality
- Viewed a demo which looked a lot better than previously
- Working to install portal software on the MEGIS server; installed the Admin part
- The vendor has asked us for a lot of input
- The project has a ways to go
- Timing = weeks not months
- Have work to do on the technical side
- Need to address shortcomings
- Hope to have a live demo by the October Board meeting
Orthoimagery Project – Larry Harwood
We have not received any imagery; batches rejected by USGS were sent back to the contractor; hope to get them by end of this year; we have loaded what we have received. In response to a question from the Chair, projected orthoimagery deliveries do not include the town of Lincoln who did receive a major parcel grant.
Barbara reported back on the work down with teachers and how very pleased they were with what was available currently on the MEGIS server.
Action Item: Jim asked that Barbara get a written testimonial from the teachers.
Parcel Grants – Larry Harwood
We have received parcel data from the contract extensions in round one and 1 town from round 2 and we are paying the bills after QAQC.
Purchasing Intermediate Products – Dan Walters
We did receive a quote of $5,000.00 and are putting together a sole source contract to purchase. The Board is acquiring this data to make available for creating additional data.
Future Agenda Items:
- Work on bond numbers and the rationale for the State Planning Office – may need to send them an adjusted list
- Need to have a discussion with action items on the county report – could change the bond request
- Need to identify strategic items
- Set priorities in October and November – get agenda out
Action Item: Carmen send the last draft of the bond request out to all members = DONE
Meeting adjourned 12:35 p.m.
Addendum # 1 GeoArchives Final Report attached below
Creating the GeoArchives:
Maine Archives of Geographic Information
A brief report presented to the GeoLibrary Board
September 20, 2006
For more detailed documentation see http://www.maine.gov/sos/arc/GeoArchives/geoarch.html
Jim Henderson, Maine State Archivist
- Retention Schedules
- Obtain GIS Layers
- Model Appraisal System
- System Documentation
- Metadata Standards
- Details Vital
- Functional Requirements
Funded by a grant from the National Historical Publications and Records Commission
Creating the GeoArchives:
Maine Archives of Geographic Information*
Goals and Objectives
This “Electronic Records Program Development Project” as defined by the National Historical Publications and Records Commission (NHPRC), was “an opportunity to test and refine the results of research and to develop innovative approaches to archival administration of electronic records.”
Several State agencies create records of permanent value on geographic information systems (GIS). The Maine Library of Geographic Information (GeoLibrary), enforces standards for, and provides public access to, many of these records. The creation of the GeoLibrary provided an opportunity to create a digital system, based on current archival research and recommended standards, to be known as the Maine Archives of Geographic Information (GeoArchives), for maintaining and providing access to State of Maine GIS records having permanent value.
The NHPRC funded our proposal, based in part on the guidelines in the Maine State Archives Digital Records Management Plan: 1999-2003 (also funded by the NHPRC), and on the expertise of Maine’s GIS professionals.
When fully deployed, the GeoArchives will insure that State and local government archival records held by the GeoLibrary will be retained permanently in the GeoArchives and will be accessible to researchers through the GeoLibrary. Eventually other archival GIS records held by State agencies, not now placed in the GeoLibrary, will be included.
This project targeted GIS data layers already in the GeoLibrary, as a controlled, standardized test universe. It focused exclusively on the data layers and specifically excluded records of publications created from that data, such as digital or printed maps. Those will be scheduled with records of the agencies producing those records.
Our strategy focused on using the proprietary production and access system used by the State of Maine (ESRI® GIS software and Oracle® databases) to integrate archival data layers into the system that manages critical GIS data: the road networks, political boundaries, hydrography, etc. As these systems are upgraded, migrated, or replaced, the archival data layers will move with the current data layers.
However, we also create XML versions of the archival data layers, both as a preservation alternative, and as a check against the current proprietary version to determine if unauthorized changes have been made.
Following the Open Archival Information System model, the following chart indicates the relationship of the GeoArchives system to the OAIS model:
"Business View" of Maine GeoArchives System
In order to meet the project goal of a sustained GeoArchives, we focused on and completed the following objectives:
- Selection of a minimum of 5 GIS data layers to be accessioned by the Archives
- Development of a model appraisal system with resulting retention schedules
- Retention schedules for the selected GIS records
- Archival data and metadata standards for creating, identifying, and maintaining the records
- Full system documentation for the GeoArchives
- Proposed technical standards to amend regulatory rules as needed
- Documentation of the project's evolution and its final product
The following provides a summary of the intended objectives. Other outcomes are discussed later.
- Retention Schedules
- Administrative records
- Functionally-based retention schedules were created for administrative records related to the development and management of GIS records.
- Constitutional and statutory provisions, judicial decisions, Executive Orders and administrative rules affecting the Maine Office of Geographic Information Systems (MeGIS) are already scheduled as archival. The State of Maine General Records Schedules already provide retention/disposition authority for most common administrative records, and specify accession of upper management level (policy-making) correspondence, minutes and agendas.
- Administrative records were examined for Maine Office of GIS, and the two organizations from which it receives direction: the GIS Executive Council and the Maine Library Of Geographic Information Board. The major functions of each agency have been evaluated for their evidential and informational value by the Archives Appraisal Committee* of the Maine State Archives.
- In addition, the Archives' Records Management Services staff assesses administrative records from other agencies that may have been involved in the creation of a layer. If not scheduled, they are scheduled and appraised for their archival value.
MeGIS staff notifies Archives’ Records Management Services Division 1) whenever a new layer has been added to the GeoLibrary, and 2) if an agency requests that a layer be removed from the GeoLibrary. The Archives insures retention/disposition schedules are legal and appropriate.
The Archives Records Management Analyst, in consultation with the directors of Divisions of Archives Services and Records Management Services, reviews
- descriptive Key Metadata Fields in the GeoLibrary catalog; and
- other relevant information sources, (e.g., that obtained by the Analyst working with agencies and MeGIS to schedule records); and
- produces a summary report recommending archival retention or destruction.
The Appraisal Committee considers the recommendations and determines if additional information is needed to make a decision. If so, Records Management staff contacts the creating agency and/or the information originator for that information. The Committee then 1) agrees to designate the layer as archival and generates a retention schedule, or 2) recommends to the Archives Advisory Board that it may be destroyed.
GIS Records Accessioned
As part of this project, the Archives has 1) appraised the following data layers as archival, 2) accessioned them into the GeoArchives, and 3) provided access to them through the GeoLibrary Data Catalog: Details are in each layer's metadata, available through the MeGIS Data Catalog at http://megis.maine.gov/catalog/.
The following are from current, active data layers in the Data Catalog:
METWP24, E911 Roads;
HYDRO, NWI, COAST, PONDS, RIVERS, STREAMS, SURF
These layers are also available in the SDE (current production) database but are substantially different from the originals, which we have accessioned, and which were digitized from the USGS quadrangles.
Model Appraisal System
The Archives will appraise each layer in the MeGIS Data Catalog on a case-by-case basis, except where a class of layers has been identified as sufficiently interrelated to be appraised as a unit. Each layer will be reviewed initially based on key Federal Geographic Data Committee (FGDC) metadata fields selected by the Archives for their likelihood of providing guidance about legal, informational, and evidential value of the layer. A custom application extracts and presents these fields to the Archives staff for each data layer under consideration. The staff creates a report assessing the metadata and offering a preliminary appraisal recommendation.
The Archives Appraisal Committee reviews the recommendation and requests more information as needed. The Committee either appraises the layer as archival and authorizes accession, or recommends a finding of "not archival" to the Archives Advisory Board. The Board, an entity independent of the Archives staff, must approve any final disposition of "destroy" for non-archival records. If it agrees with the "not archival" recommendation, the owner of the data layer and MeGIS will be notified of that status.
If either the Board or the State Archivist determines that the data layer is archival, destruction is not permitted. MeGIS will be notified of the layer's archival status and the Archive will proceed to accession it.
GIS layers are held in MeGIS servers that provide access through a Data Catalog. Layers appraised as archival, along with other associated data tables, are copied to a read/write server for processing. Processing includes assessing existing metadata, supplementing it if necessary, and adding Archives' metadata defining ownership and access procedures.
A copy of the layer, with Archives metadata and associated data tables, is sent in native format to a read only table space, which provides access through the Data Catalog in the same way MeGIS active layers are queried.
A second copy in XML, with Archives metadata and associated data tables, is sent to another server in a read only table space, which provides archival storage.
- GIS databases in the GeoLibrary are created using ESRI's ARC/INFO or a fully compatible through a common data interchange format.
- GeoLibrary layers are held in Oracle databases in read-only table spaces.
- Some GeoLibrary layers are stored and edited in ESRI's ARC SDE format.
- The GeoArchives data layers are held in Oracle databases in read-only table spaces, both in native format and in XML.
- Less expensive "Slow Storage" technology hosts the data "Based on our tests, "slow" storage causes no detectable degradation in retrieval time.
These provisions support the functional requirement insuring the "soundness" of the GeoArchives data layers.
- All GeoArchives data layers are created as read only, and in ESRI’s default "non-editable format."
- Oracle’s “Auditor” function is active for all layers within the GeoArchives account
- Backups will be taken daily, with a five-week rotation of daily tapes, Saturday through Thursday. Friday night backups are kept in a 26 week rotation.
- Periodic comparisons of an original XML version with a current XML version will insure data integrity. (See "soundness" in Appendix A for details.)
Archival data and metadata standards
The data layers, once appraised as archival, are accessioned as created by the responsible agency. The "Data Standards for Maine Geographic Information Systems 2002," adopted by the Information Services Policy Board (now defunct) still stand as the required standards for GIS data in the GeoLibrary, and by extension the GIS data accessioned by the Archives. The archives will continue to monitor standards for the development of GIS layers wherever they reside.
The GeoLibrary effectively monitors the documentation of data quality through accompanying FGDC metadata. Archival metadata standards have been developed to document the Archives accession, ownership, and management of the archival layers.
Movement of Layers from Production to GeoArchives
Proposed technical standards
The Data Standards for Maine Geographic Information Systems insures strong compliance with, Federal Geographic Data Committee's standards for data and metadata. Instead of amending those standards, the Archives finds that its best approach is to 1) insure that archival layers contain the required data and metadata, and 2) add archival metadata following the FGDC standards.
Several unanticipated benefits have flowed from the execution of this project. Not the least of these has been higher visibility and credibility of the Archives among State information technology staff and managers.
The project has required the Records Management Services Division of the State Archives to apply a functional approach to scheduling GIS administrative records, something it has done only once before. The experience will have benefits for the development of future functional schedules.
The Maine Office of GIS has reassessed its security measures and data integrity efforts because the Archives has raised these issues in an attempt to insure the survival of the archival GIS layers.
One of the greatest unanticipated benefit is the contribution of this project to the development of a comprehensive Digital Archives Plan for the State Archives.
Impact on the Maine State Archives and Others
Maine Digital Archives
Based on its earlier Digital Records Plan and the GeoArchives experience, the Archives has drafted a much more comprehensive Digital Archives Plan. The latter is a comprehensive statement of the preservation needs and potential strategies regarding born digital and digitized analog records.
Born digital records obviously includes GIS records as well as other databases, office suites (wordprocessing, spreadsheets, etc.), digital audio and video, imaging systems, and e-mail. Digitized records (by State agencies or the Archives for access purposes) include scanned paper or microfilm and conversion of motion picture film, analog audio and video tape to digital format.
While the project met its major objective of developing a GeoArchives system, we are constantly reminded that the devil is, indeed, in the details. The Archives staff must be vigilant in insuring that
- communications with the Maine Office of GIS be clear and continuous,
- documentation of applications supporting the GeoArchives is complete,
- documentation of the location and physical controls on servers hosting archival layers is complete, and
- metadata for layers accessioned by the Archives is complete and consistent with the GeoArchives Manual.
The GeoArchives continues beyond the project period. Two Archives staff members, one from Archives Services and one from Records Management Services, are specifically assigned, on a part-time basis, to the appraisal and accessioning functions.
Appendix A: Management for Functional Requirements
The project has adopted a set of functional requirements for recordkeeping systems to insure the permanent retention of the data layers.
Insure that the system and rules created to maintain the GeoArchives operate as designed. An initial and periodic test of accessioned archives data layers will provide operational evidence of system compliance. Archives staff will access and test each layer within one month of accession, and on a six-month schedule thereafter. The test will include
- Downloading the layer using instructions on the GeoLibrary site
- Displaying a map and printing it; adding themes from other archival layers
- Documenting the test and retaining the documentation for future reference
- Conducting annual e-mail users surveys regarding access and use
Insure the GeoArchives has policies, assigned responsibilities, and formal methods for effective management. The Director of Archives Services will report annually on June 30th to the State Archivist on the status of the GeoArchives and the status of each of the functional requirements.
The GeoLibrary and Geo Archives must monitor the quality of information being input and insure that information is accurate, documented and consistent with policies. In appraising each data layer, Archives staff will review the "Source Information" metadata field to insure it contains sufficient data quality and data accuracy information. Archives staff may consult with the GIS Metadata Coordinator for assistance in interpretation and compliance.
Records incorporate or link to information about the context of their creation, e.g., the relevant administrative records. Archival metadata will include reference to scheduled associated records and how to obtain them in the FGDC "Other_Citation_Details" field. Archives staff must review this metadata each time a version of a data layer is accessioned.
The system must validate records creators and/or authorizers to insure information is authentic. Archival records must be trusted as authentic, unchanged from their status upon accession. The Archives must be confident that the data layer was created by the stated "originator" and that people authorized to make changes are clearly identified and audited. Archives staff will document the process by which layers are received and "published" by the GeoLibrary, including how it verifies the authority of the person acting for the originator. This is integral to the Appraisal Process outlined in Appendix B.
Record integrity is protected from accidental or purposive damage or destruction and from any modification after they have been placed in archival status.
The Archives' policy mandates that archival layers will not be changed once accessioned, thus no general authority for change will be issued. If a layer is "changed" in any way, it will be accessioned as a new archival layer. In order to insure record integrity, the following procedures will be followed for each data layer accessioned:
- The Archives, using tools developed by MEGIS, copies a data layer appraised as archival from the Geo Library to an Oracle-based read/write GeoArchives processing table space.
- Archives staff conducts the Accession Process, outlined in Appendix B, in a read/write table space.
- The data layer is copied to the read only GeoArchives Storage table space as a read only table within that space.
- A copy of this data layer is exported as an XML file to
- an Oracle Test space in a UNIX server, and
- a GeoArchives Test space on a separate UNIX server
- On the first work day of each month, ARC Catalog® converts all GeoArchives data layers to XML files and places one copy of each in each of the spaces noted in 4.
- In each of the spaces, using the UNIX DIFF command, the current month XML copy will be compared to the original XML copy created when each data layer was first placed in the GeoLibrary.
- If "no difference" is reported for a data layer in each of the two test spaces, the current month XML copies are deleted from the test spaces. (The original XML copies are retained.)
- If a difference is reported in either space, Archives and MEGIS staff will research the reasons for the difference, and determine remedies, including
a. If the original XML copy appears to be corrupted, restore it from the GeoArchives Storage copy.
b. If the GeoArchives Storage copy appears to be corrupted, restore it from the original XML copy.
- Documentation of each report finding a difference will be included in the permanent records of the Maine State Archives. The State Archivist must approve all remedial actions, and certify in the documentation that the data and metadata in the affected layer is unchanged from the original, with documented exceptions if necessary.
An initial test of this approach was made on December 20, 2005:
- This morning I tested creating 2 .xml exports of GIS data layers and compared the 2 .xml files using the Unix diff command. I used ArcCatalog (the viewer that Archives is using), logged in with the MEARCHIVES database account, right clicked the MEARCHIVES.Metwp24L_06302005 layer, selected export, and exported to .xml format (onto the MEGIS Unix server). I named the first export Metwp24L_06302005a.xml and created a second export named Metwp24L_06302005b.xml. I then ran the comparison of the 2 files on the Unix server using the Unix diff command. The diff command did not find any differences between the 2 files. I then tried the same process on Windows. I created 2 .xml files (using the same process previously listed) and used the DOS command fc to compare the 2 .xml files. The DOS fc command did not find any differences between the 2 files. (Maine GIS partner for the project.)
Unix syntax: diff Metwp24L_06302005a.xml Metwp24L_06302005b.xml
Dos syntax: fc Metwp24L_06302005a.xml Metwp24L_06302005b.xml
Ensuring Data Layer Integrity in the Maine GeoArchives
GeoArchives' controls must preserve auditability of interactions external to the system (such as during media migration or transfer). The Maine Office of GIS will develop a description of the procedures for auditing "external interactions' for the GeoLibrary. This description will be scheduled as archival and Archives staff will review the description for significant changes annually.
The GeoArchives system must document all logical archival records it contains, indicate the terms under which they are available for research, and retrieve them for authorized users. GeoArchives data layers have been clearly identified in the GeoLibrary catalog and will be available for use. Related records will be scheduled, identified, and will be available through the Archives or the creating agency.
Record content, structural representation and representation of context must be exportable, in standard protocols. Archival data layers will be exportable in a variety of commonly used GIS formats and in XML. Archives staff will confirm these formats with the GeoLibrary annually.
The GeoArchives system must present records to allow views in effect at the time any record was in existence. Data layers for "snapshots" will be accessioned a) for layers updated annually or less frequently, each time a new version is presented to the GeoLibrary, or b) for frequently updated layers, such as E911 roads, at least semi-annually. The new version may include several changes made by the creating agency in preparation for the new publication. Researchers may combine several archival versions in a particular data layer series as "themes" showing the views of the data at different "snapshot" dates.
The GeoArchives system must support delivery of redacted, summarized, or censored copies and keep records of the version released. Current data layers contain no confidential records. The problem has not been addressed at this time.
Archive Snapshot instructions
This process is to be used in Arc Catalog, in conjunction with the ArchiveOMatic2 menus and functions.
Copy the data to the Archives account
Start Arc Catalog, double click “database connections,” then double click "Archives.sde" to open the Archives account.
Locate the data to be archived and select it by clicking once on it. In some cases (such as with Metwp24) the data is located within a feature dataset. In that case, open the feature dataset, then select the feature class to be archived. Note: make sure you do not select data that has already been archived!
Press the ArchiveOMatic2 button on the Arc Catalog toolbar. This will open a menu. The menu will display Archive Determination Date, Accession Date, input dataset name, and output dataset name. The dates and output name can all be changed by the user if needed. By default, the dates are set equal to the current date, and the output name will be MEARCHIVES.<layer name>_<date>.
If this is all correct, click the “Copy to Archive account” button, otherwise click the Cancel button. When the Copy button is clicked, the data will be copied to the Archives SDE account with the new name (oldname_mmddyyyy). This process generally takes a few seconds, longer for large datasets.
When finished, a small popup message will indicate that the data layer has been copied.
Addendum # 2 Discussion on the County Report
Overview & Objectives
- Requirements analysis and implementation framework
- For enhancing geospatial technology activities beneficial to ME county govt.
- Increasing county utilization of geospatial technology and data
- Establish/strengthen “regional service center” concept to aid counties
- Identify funding mechanisms to provide baseline support to counties
- Adding some details that were not covered in the scope of the 2002 Strategic Plan to Develop the GeoLibrary study
- Requirements and Information Gathering Sessions
- Education and stakeholder input forums
June, 2005: Gorham
July, 2005: Orono
- Project progress briefing to County Commissioners
- Findings presentation and refinement
- Analysis and Functional Framework Identification
– Coming to consensus on a workable model for ME
- Implementation Strategy & Benefits Identification
– Identifying details of a program to meet identified needs and
to maximize the benefits to be gained
- Identification and Assessment of Funding Options
Nationally: GIS & County Government
- 3100+ Counties Nationwide
- Most (75%+) using GIS in some form
- In many regions serve as the primary “local GIS” practioners
- Primary functions:
- Land ownership records + parcel information
- Road planning + maintenance
- Utility planning + maintenance
- Comprehensive planning and zoning ordinances
- Parks and recreation planning
Maine: GIS & County Government
- 16 Counties
- Most using some GIS, but in very limited ways
- Core mapping functions:
- Emergency management applications
- Law enforcement
- Basic location and event mapping
Observations from Stakeholders (Political & Strategic)
- Regionalization can work for GIS, as it is working for schools
- Lincoln Co: 19 towns, 6 school districts, 1 GIS system?
- County GIS must server both data consumers and producers
- Applications should not be “read only” viewers
- Large value aiding regional data capture, e.g., for E911 road enhancements, critical infrastructure, etc.
- County GIS data coverage should be able to cross state and international borders
- Counties can aid state efforts at data standardization
- Counties can help “standardize” data sets such as parcels
- Counties are “caught in the middle” between state and munis
- Perceived home rule barriers to “outsourcing” to the county
- TTWWADI Inertia – “that’s the way we’ve always done it”
- Natural reluctance to change/modify existing work patterns
Observations from Stakeholders (Technical, Educational, & Funding)
- GIS technical support remains a requirement
- GIS is easier than ever, but remains technically challenging for county govt.
- GIS technical expertise can be shared
- Better to have 1 full-time person covering 10 towns than 10 people doing
- GIS 10% of the time
- Sharing of technical infrastructure is feasible
- Data servers and devices like plotters can be easily shared
- High-bandwidth networking and internet continues to expand
- E.g., Aroostook and other border counties have gained high-speed internet via HLS funding
- Enables web-based deployment of GIS capabilities
- Counties should adopt existing state and federal standards whenever possible
- Staff resources are extremely limited
- Unlike other parts of the country, county staff is very limited
- New budget expressly for county GIS is extremely unlikely
- Creative financing via state and federal sources (e.g. GeoLibrary, USGS/FGDC)
Observations from Stakeholders (Core Applications: Deed Registries)
- Observations & Requirements
- Move to increased automation and scanned document management is strongly under way
- Will increase computer capabilities, and GIS is a logical follow-on to these efforts
- New England property management has some systemic inefficiencies
- Deeds and surveys done by County
- Assessments and parcel maintenance done by towns
- Enlightened jurisdictions can move beyond this with proactive coordination
- Standardization of data capture protocols can aid in automation efforts
- Requirements for digital submission of surveys/plans
- Requiring municipal Map-Lot numbers on deeds
- Perceived threats from private sector data providers
- These companies can undermine registry’s historic role as data provider and can impact data access revenue streams
- Limits county interest in, and ability to fund further automation
- “Deed Registries cannot be everybody’s cash cow”
- Many initiatives have coveted the property transfer tax
- In Massachusetts, temporary “technology surcharge” on registry recording fee generated significant revenue to aid in registry automation efforts.
Observations from Stakeholders (Core Applications: EMAs)
- Observations & Requirements
- A couple of county EMAs are actively involved with and are using GIS
- Cumberland and Somerset counties are examples -
Achieved through individual vision/effort and creative financing
- Emergency response requires consistent, dependable data
- On critical infrastructure, on sensitive populations
- Creates a “common operating picture” for all levels of government
- Flood event of May, 2006 provided some challenges that GIS could have helped with
- Who to evacuate first?
- What roads are susceptible to flooding? Where do detours go?
- Codify lessons learned into the GIS database for “next time”
- Emergency Management GIS needs to be able to function in a disconnected fashion
- Local copies of key databases to support emergencies where the internet may be unavailable
Functional Framework for Increased County Use of GIS Technology
- Standards: will help create regional data that can roll-up to state
- Data: enable local collection/validation of data
- Applications: web-based applications served to county users
- Technical Assistance: leverage expertise across wider areas, perhaps through regional service centers.
- Privacy Protection: Need to be sensitive to concerns; use technology to address concerns
Increased Access to
Geospatial Intelligence + Analysis
Tech Help and Outreach
Real World (un-analyzed)
Implementation Plan (Ideal Scenario – money is not object)
- Technical Infrastructure Provided to Counties
- High-bandwidth internet
- GIS ready computers
- Access to desktop GIS software licenses
- Standards Implemented and Enforced to Facilitate GIS Implementation
- Registry records, surveys and parcels
- Critical infrastructure and sensitive populations
- Active Data Automation Program
- Direct state support of property/parcel automation and HLS data
- Application Development
- Web--based application targeted for county users
- E.g., Distributed, web--based parcel data maintenance
- Technical Assistance
- State supported regional service centers
- Providing both help and direct capabilities to “have nots”
- In reality, this is the long-term goal. Near term efforts will hopefully advance the state towards these goals.
Implementation Path (Practical Reality)
- Achieving widespread county utilization of GIS will be difficult and will require significant investments.
- Challenge is to start a long-term process with a realistic level of investment
Modest two pronged approach:
- Create a MeGIS County Liaison position
- Actively build relationships with counties
- Help counties to fully leverage what MeGIS has to offer
- Stories of county personnel being unaware of what was available
- Identify county application specifications
- Identify and work to gain 3rd party funding
- Develop pilot projects
- Coordinate ongoing initiatives within the county and the GeoLibrary
2. Establish Regional Service Centers
- Envisioned in the 2002 Strategic Plan
- Not just for counties, but also for municipalities
- Many deployment options/questions
- How many? Where to put them? Geographically? Organizationally?
- Requires funding and commitment
- Many potential sources exist
- It is most likely that progress will be made through a combination of funding sources
1. Seed money from County Govt.
- $2,000 /year/county = $32,000
- County Commissioners were “receptive” at Augusta meeting
2. Grant money from Federal partners
- USGS and DHS are actively providing grant money for GIS
3. Transfer tax revenue
- It may be difficult, but it’s not impossible
4. County Pilot Project funding
- Aroostook is funding a pilot effort with LURC and Maine Revenue Service for parcel-deed linkages
5. GeoLibrary starter funds
- Funding for Regional Service Centers would add value to county efforts
6. Regionalization funding
- Grants through Maine Development Foundation for consolidating services across jurisdictions
- GIS could be a component of “consolidation planning” and implementation