Usually, one of the most common questions during water supply GIS software procurement process from water utilities is that how do they import their existing network information from old documentation and other data sources to KeyAqua. In following sections, we cover most important things what water utilities should know about the data import to GIS solutions.
Often water utilities have existing water- sewer- and stormwater network information in some format. This can be old paper maps, AutoCAD-files, Excel-files, PowerPoint-files, existing GIS system, third party IT-system data dumps or combination of these. Question is how to import all relevant and up to date data to new KeyAqua GIS system. All data can be imported manually by digitizing network from scratch and using existing documents and files as a reference. KeyAqua has all necessary tools for users to implement manual documentation with data quality checks. Usually, this method requires quite much manual labor and most likely is not most cost-efficient way. Another very common way is to implement programmatic data migration where big amount of network data can be imported relatively fast when original data quality is good enough. Next section we cover most important things related to these methods and what good data quality means.
Manually digitizing network information usually consumes lot of time due to big amount of network information which needs to be digitized one duct, node and annotation at the time. Benefit is that documenting personnel can use these several existing data sources to check data accuracy relevancy, decide what is the solution based on each part of the network and then digitize high quality result on GIS system. Quality of manual documentation result is very good because when inserting data manually, user can decide exactly where to put objects and related information, for example annotation locations etc. This is something that programmatic data migration tools and methods cannot make.
Often manual digitizing is used when existing network data quality is very bad, water utility has several overlapping documents and files with different versions and sometimes also older water utility retired personnel needs to be consulted how and where network was built long time ago.
Programmatic data migration method is often fastest and most cost-efficient way to import big amount of network data to GIS solution. This method requires that quality of existing documentation, regardless of the format, is high quality (or even decent quality). Good data quality means that data management professionals, computers and special data manipulation programs like FME are able to read the data and possible network object relations included.
What ever the data format is, it must be in coherent format and documentation logic cannot change randomly within same file or document. It does not matter if data is in AutoCAD, Excel, Shape, any other electronic format or combination of all of these. Most important part is that it is possible to create data import logic and this is why data must be in coherent format so that data manipulation programs can tell the difference and similarities between all water network ducts and nodes with attribute information. Also, data management professionals who create import logic must understand how network is documented in deep detailed level. In ideal case water utility should provide network documentation reading instructions but usually this is not possible and no such documents exists.
If several different data files need to be combined, for example duct locations are in AutoCAD file and duct attribute information is in Excel-file, there must be identification number or identical coordinates to tell data manipulation programs that which duct, and which attribute information is linked together. Worst case water utility's existing network data is not in coherent format and there are no links between different data sources and documents. In these cases programmatic data migration cannot be used on (or it can be used only partially to import data) but good end result requires more or less manual documentation work.
Levels and methods how water utilities document their network are usually always different and it is possible that data migration professionals misunderstand for example symbols or object relations and create false import logic. This is why open and active discussion between water utility and service provider is needed to remove all ambiquities when migrating data and to ensure best results.
It does not matter how small or big the water supply network is, computers can import data fast after migration logic is created. Biggest amount of work is to create migration logic based on existing data model so that computer can understand how to handle and import data. Network size basically has no effect on amount of work needed.
When implementing manual or programmatic data migration, it is possible to improve the quality of network documentation using different manual and automatic methods. If you want to learn more, these methods are covered in data quality improvement blog here. Also, if you are interested how to import other kind of data like customer and consumer point information from third party IT-systems, please read blog post about interfaces and integrations here.
Now when existing network data import is ready, we can move to surveying and collecting missing data from the field. Many times, water network owners are missing some part of the network documentation totally. Usually this is related to oldest part of the network when creating documentation was not considered so important.
Field surveyors can gather network data directly from the field using professional high precision GPS field survey devices. Nowadays it is also possible to acquire high precision GPS antenna and connect it to water utility's existing tablet. This means that you can get location information from your ducts and nodes in few centimeters’ accuracy at best. When all necessary data is collected, it is possible to export this information from survey device as text format which includes network object’s coordinates and type. This produced text file can be uploaded directly to KeyAqua and system will automatically create network. Another way is to export this surveyed information as cad-file from the survey device and import file directly to KeyAqua as a background picture as reference layer. This phase requires manual work to snap the file's objects when digitizing and create real network objects.
You can also use KeyAqua mobile interface directly with your smartphone or tablet and gather information from the field by taking pictures and adding notes etc. This information is uploaded directly to KeyAqua and no other manual steps are needed.
Want to know more about KeyAqua GIS? Click here to see more.