Advanced Configuration of Processing Jobs in WebShare Enterprise

Overview

This document describes advanced configuration of processing jobs in WebShare Enterprise. You only need to consider these options if you encounter frequently failing jobs. Also please check that your server fulfills the system requirements for WebShare Enterprise.

We refer to long-running CPU-intensive jobs as “processing jobs”. Every user with the Domain Administrator role can see all current or previous jobs in the Job Queue Administration UI.

General Guidelines

• All configuration files mentioned here use the YAML file format.
• Lines starting with a hash character # are YAML comments, and don’t have any effect.
• To edit a file with root permissions on the commandline, you can use e.g. the “Nano” editor: $ sudo nano <FILE>
• Any changes made to these configuration files will be reverted once you install an update of WebShare Enterprise. Therefore it is recommended to take a note of your changes, in case that you want to apply them again later.

Point and Scan Exports

Currently, always up to two jobs of each type (point or scan export) run in parallel. This limit is not configurable. This limit does not include jobs that have crashed and are stuck in “running” state. If needed, the administrator can configure the size limit and reserved disk space for point and scan exports. To do so, proceed as follows:

1. Open /opt/faro-webshare/etc/template/docker-compose.template.yaml using an editor with root permissions.
    
2. Replace all occurrences of --size-limit 100 with your preferred size limit for exports in gigabytes.
   • E.g. use --size-limit 123 for a limit of 123 GB. If the exported file is larger than this limit, the export will abort. In some cases, the user may still be able to download the partial export up to this point.
      
3. Replace all occurrences of --reserved-space 2 with your preferred reserved disk space in gigabytes.
   • E.g. use --reserved-space 10 to make sure that the exporter won’t touch the last 10 GB of the disk. If the free disk space during a point or scan export is below this limit, the export will abort with an error. In some cases, the user may still be able to download the partial export up to this point.
      
4. There are some further options that you can change if needed:
   • --thread-factor 2
     Number of threads = 2 * number of CPU cores. If your CPU is overloaded, choose a smaller value.
   • --creation-speedup-factor 1
     Trade-off CPU vs. memory usage. Small value = Long computation time but lower memory usage. Big value = Short computation time but higher memory usage. It is recommended to choose a value between 1 and 50.
   • --purge 7
     Delete each exported file after 7 days. Change according to your user’s needs.
      
5. After you have saved your changes to this file, restart WebShare Enterprise using: sudo faro-webshare stop; sudo faro-webshare up -d

Point Cloud and Orthophoto Creation

By default, up to two jobs of each type (Orthophoto or Point Cloud) are run in parallel. This limit includes jobs that have crashed and are stuck in “running” state. Use the Job Queue Administration UI as described in the user manual to cancel or restart such jobs. To configure Orthophoto and Point Cloud job processing, please proceed as follows:

1. Open /opt/faro-webshare/etc/template/webshare_ws.template.yml using an editor with root permissions.
    
2. If you want to change the maximum number of jobs running in parallel, update the value of:
   wsworker_orthophoto.max_parallel or wsworker_projectpointcloud.max_parallel
    
3. If you want to change the maximum RAM usage, in particular of the point cloud builder, update this configuration parameter:wsworker_projectpointcloud.max_heap or wsworker_orthophoto.max_heap (for the Orthophoto builder).
    
4. In the lines below, only one of the properties ("mb", "percent_total"  or "percent_available") should be enabled; the other two should be commented out. (Lines starting with # are “commented out” = disabled.)
   • mb
     Configures the absolute memory limit in megabytes.
   • percent_total
     Configures the memory limit relative to the total RAM of your server (percentage).
   • percent_available
     Configures the memory limit relative to the currently available RAM of your server (percentage). The available RAM is measured when the job is started.
      
     • It is recommended to conservatively configure the memory limits. This means that you should keep enough RAM left for the operating system and WebShare Enterprise in general. If you allow the processing jobs to consume too much memory, your server may become unresponsive and need a reboot. Here are a few example configurations.
        
     • Example 1: Run max. 1 point cloud job in parallel, each job using up to 5000 MB of RAM:
       wsworker_projectpointcloud.max_parallel: 1
wsworker_projectpointcloud.max_heap:
     mb: 5000
     #percent_total: 10
     #percent_available: 20
     • Example 2: Run max. 4 point cloud jobs in parallel, each job using up to 10% of your total RAM:
       wsworker_projectpointcloud.max_parallel: 4
wsworker_projectpointcloud.max_heap:
     #mb: 5000
     percent_total: 10
     #percent_available: 20
     • Example 3: Run max. 2 point cloud jobs in parallel, each job using up to 20% of your available RAM:
       wsworker_projectpointcloud.max_parallel: 2
wsworker_projectpointcloud.max_heap:
     #mb: 5000
     #percent_total: 10
     percent_available: 20
        
5. After you have saved your changes to this file, restart WebShare Enterprise using: sudo faro-webshare stop; sudo faro-webshare up -d