Data Backup and Recovery

<none>

MobileMap provides several methods for automated and manual data backup and reocvery.  These tools are intended to prevent data loss in the event of a mobile device failure.  They are not intended to provide offline workflows for data management, but can be used to help protect data prior to data upload when data upload will be delayed due to lack of internet access.

Settings

Backup Interval - Automated data backup is currently only enabled for MobileMap Cruise, as it is triggered by save of specific cruise features such as stands, plots or trees.  If not using cruise functionality, use manual backup to save data.  If automated backup of cruise data is desired, enable this in MobileMap settings by setting the backup frequency (Settings > MobileMap Cruise > General > Database Backup Interval) to the appropriate interval (e.g., every plot save operation, every stand save, every 10 trees, etc.)

Backup Format - The default backup format is a single text file, containing JSON data which includes only the new and modified features (see description below).  An alternative SQLite database format is also available (see description below) and can be enabled in settings (Settings > MobileMap > General > Data Backup in SQLite Format).)

Backup File Location

All backups will be stored in an 'Export' folder in the selected Data Directory.  MB&G recommends the use of MicroSD cards for MobileMap base data and thus for the Data Directory.  By using MicroSD card, automated backups will be stored on external media that can be removed is a device is damaged in the field.  When automated backups (MobileMap Cruise only) are used with a MicroSD card, it is possible to recover all data collected during the day even if the mobile device is damaged at the end of a day (e.g., dropped in a stream) by removing the SD card, drying the SD card off and inserting it into a PC, accessing the most recent backup file (e.g. JSON text file) and uploading the new/modified data to the feature service by cutting and pasting into a browser.

Export Feature Data

In addition to the automated backup described above, data can be backed up manually (Actions > Export Database).  This may be necessary in situations where a device is damaged and no longer able to access the internet (and thus cannot upload) or if upload errors cannot be resolved.  One situation where this can occur is when features have been deleted on the server (e.g., by InventoryManager) then a MobileMap users modifies those feature and tries to upload their changes.  In this case, the change cannot be applied, because the feature does not exist in the service.  It is possible to export these data and manually load them into the service. 

Export of JSON data:

This approach is the preferred (default) format for automated and manual backup of data from MobileMap.  This approach exports only the new/modified feature data and stores it in a single text file that contains a JSON representation of the Esri feature data.  It includes the links to the feature service endpoints for adding/updating data for each layer that contains new/modified features.  While it is intended for use by data managers, standard users can be trained to use this approach.

Because this approach exports only new/modified data, there are scenarios where the JSON files produced will not initially include the desired data.  Consider the case where a user successfully uploads data from MobileMap, then deletes those data using AGOL, InventoryManager, etc., but decides at a later time that they want to recover those data from MobileMap.  In this case, MobileMap will not see those data as new/modified, since they have not changed since they were uploaded.  One way to address this issue is to re-open and save each feature that needs to be recovered.  Simply opening and closting the tree list for each plot, for example should re-save the plot and all trees, and thus include these plots and trees in the JSON export file.

When data are exported to JSON format, they will be saved in the data directory's Export subfolder, in a file named MobileMap_XXXX_todays_date_incremental_backup_JSON.txt.  To load these data into the feature service, copy this file to a PC, open it using a text editor, and then load the data by copying in the JSON representation of the data into the matching feature service layer endpoint.  For example, if the file contains the following text:

Layer: Plots

Updates: 1
URL: https://services1.arcgis.com/ABCDEFG/arcgis/rest/services/Data_Model_v1/...
[{ "geometry" : {"x":-1.271745906241922E7,"y":5788956.715272526,"spatialReference":{"wkid":102100,"latestWkid":3857}}, "attributes" : {"Archived":null,"Big_BAF":"None","Created":1573845358349,"Created_Source":"MobileMap 4.3.7-standard","Cruiser_ID":"KQH","Design1_Size":"BAF20","Design2_Size":0.0,"Edited":1573866200609,"Edited_Source":"MobileMap 4.3.7-standard","OBJECTID":11,"Parent_ID":"{4743dc00-abcd-44c1-9bee-7a748ec24811}","Plot_ID":1,"Plot_Type":1,"Sale_Name":"Big Sale","Sample_Design":"BAF","Sample_Type":"None","Status":"Cruised","Unique_ID":"{aedd61a9-5f21-43ba-9981-71d75a359528}"}}]

To load the data above, open a browser and past into the address bar the URL (https://services1.arcgis.com/ABCDEFG/arcgis/rest/services/Data_Model_v1/...).  You might be prompted to log in if you are not logged into the site.  Next, paste in the data text string, which will start with [{ and end with }].  Click submit and note the success or error message(s). 

If the upload from MobileMap failed because the features had been deleted from the server, they will fail when attempting this process as well.  If you wish to re-create these features ni the service, change the URL from .../updateFeatures to .../addFeatures and then repeate the process to save these as new features.  

If the update (or add) failes due to a validation error, you may need to use a text editor to modify your data.  Consider the case where a feature service hosted in ArcGIS Portal was updated (overwritten) by a data manager and a data type was changed from Text to Integer.  The JSON data will still contain text values like "small", "medium", "large" while the new service will expect values like 1, 2 or 3.  Before the data can be saved in the service, they will need to be manually fixed to address this disconnect.

Export of SQLite database:

This approach produces a full SQLite database (all data, even data that has been downloaded from the feature service and is unchanged).  This approach can be useful if a large amount of data must be recoverd and the data have not been modified since they were uploaded.  In that case, they would not initially be included in the default JSON export format, and using the SQLite format would avoid having to re-save all features just to get them to export.  The SQLite format, however, is not a standard Esri offline geodatabase, it is a MobileMap-specific SQLite format.  This approach is for expert users as it requires knowledge of SQL/SQLite, and a SQLite viewer/editor (e.g., DB Browser for SQLite https://sqlitebrowser.org/).  MB&G can provide instructions for recoving data from this SQLite file to data managers if needed.  MB&G can also perform this data recovery as a service upon request. 

Export Cruise Data

Export Cruise data to CSV files:

Similar to the Cruise Export functionality in InventoryManager, this approach exports cruise data as 3 CSV files containing the attributes (not spatial data) for Stands (Cruise Units), Plots, Trees.  This approach is typically used by users who want to transfer to a PC to run Excel functions for QA prior to data upload to the feature service.  This approach requires an export conifiguraton file (same format as the file used by InventoryManager for cruise export).  This file should contain all of the fields that need to be exported.  The export configuration file must be named 'cruiseExportMapping.csv' and must be stored in a folder named 'Export' that is within a MobileMap project folder on your device (e.g., SD card > MobileMap > MyProject > Export > cruiseExportMapping.csv).  More details on the format of export configuration file file can be provided upon request.