Optimizing BuildPDFFromDocuments steps

The BuildPDFFromDocuments step is a resource-intensive step that assembles a PDF job from documents that have been identified by RICOH ProcessDirector. If there are a large number of documents or if you use complex rules in your file, the processing duration for this step is measured in hours, not minutes. Advanced configuration options are available to reduce processing time in this step for very large or very complex jobs.
    Important:
  • This configuration is highly advanced. It is not recommended for all RICOH ProcessDirector implementations.

Your system is a good candidate for this configuration if all of these are true:

  • Processing a single PDF job faster is more important than total throughput.
  • Your PDF jobs contain many documents or use complex rules that cause processing to take a very long time.
  • You have (or can acquire) adequate hardware resources for the system that the step uses. The system must have many CPU cores, as each thread used for processing requires three or four background threads to manage memory usage and other activities. The system must also have at least 16GB RAM available for this processing on your primary computer or additional servers to set up as secondary servers.
  • Members of your team have the skills and tools necessary to monitor system performance. To tune the system correctly, you must be able to monitor:
    • Memory used by Java processes
    • Java garbage collection
    • System memory usage
    • CPU utilization
    • Resource swapping
    • System I/O

In this configuration, you set up your system to run the BuildPDFFromDocuments step using multiple threads. The configuration requires the step to run in a separate Java Virtual Machine (JVM). To run the step in its own JVM, create a secondary server and tune the BuildPDFFromDocuments step to run on that server. No other steps should be configured to run on that server.

To optimize BuildPDFFromDocuments steps:

  1. Determine whether your primary computer has sufficient memory to support this configuration.
    Review the topic Tuning Java memory allocation. We recommend allocating at least 16GB of RAM to the local secondary server. If your calculations from that topic indicate that you have 16GB RAM available or you can install additional memory on the system, you can use a local secondary server. Continue with step .
  2. Create a local secondary server using this procedure: Defining secondary servers on the primary computer.
    Set the In general server pool property to No and leave the Maximum resource intensive step set to 1.
      Note:
    • Stop after you create the secondary server. Do not tune the BuildPDFFromDocuments step template yet.
  3. Update the JVM settings for this secondary server.
    1. Open %AIWDATA%\config\jvmsettings.cfg in a text editor.
      By default, %AIWDATA% is \aiw\aiw1.
    2. Copy the line for the primary server that looks like this:
      primary=-Xmx2048m -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true

      Paste it into a new line in the file.

      The value after primary=-Xmx is the maximum amount of heap memory the RICOH ProcessDirector Java run time environment is allowed to use for the RICOH ProcessDirector primary process. In this example, the primary server can use 2048MB (2GB) of RAM for its heap.

    3. Update the line that you copied to change primary to the value of the Server name property for the secondary server.
    4. Update the value after secondary_server_name=-Xmx to the amount of memory that you have available for this process. The value should be at least 16g.
    5. If the line for your secondary server does not include this setting: -XX:+UseG1GC, update the line to add it.
    6. Save and close the file.
  4. Restart the RICOH ProcessDirector service to apply the new settings.
  5. Enable the secondary server in the RICOH ProcessDirector interface.
  6. Tune the BuildPDFFromDocuments step template.
    We recommend making a copy of the step template to use in the multithreaded configuration.
    1. Open Workflow ⇒ Step Templates.
    2. Find the BuildPDFFromDocuments step template, right-click and choose Copy.
    3. On the General tab, enter a value for the Name property.

      Choose a name that clearly shows that this is the multithreaded version of the step.

    4. On the Tuning tab:
      • Set Advanced tuning to Multithread this step.
      • Set Servers to use to Run on specific servers, then select only the secondary server you created.
    5. On the PDF tab, look at all of the Build PDF control file properties.
      • If your step uses one or more control files, verify that the Build library is set to Use default or control file.
      • If your step does not use any control files, set the Build library to PDF Java Toolkit.
        Note:
      • The values on the PDF tab can be different for each workflow. Update these values as needed after you add this step template to a workflow.
    6. Update any other settings as needed for your installation.
    7. Click OK.
  7. Enable the step template.
  8. Set the maximum number of threads that the step template is permitted to use.
    • Start with a number of threads that is approximately 25% of the cores in the machine. Each thread used by this step requires other threads to support it, so you cannot use all of the threads for the step. For example, if you have a 16 core server, allocate 4 threads to this step.
    • You might have to experiment to determine the optimal number of threads for your installation.
    1. Open %AIWDATA%\config\product.cfg in a text editor.
    2. Add this line to the bottom of the file, substituting the number of threads you want to use for number_of_threads: 
      useBuildPdfMT_MaxThreads=number_of_threads
        Note:
      • We recommend starting with 4 threads.
    3. Save and close the file.
  9. Restart the RICOH ProcessDirector service to apply the new setting.
  10. Test a workflow with the new step.
    We recommend copying the workflow whose performance you are trying to improve, then replacing the current BuildPDFFromDocuments step with the multithreaded version. Run a representative job through the workflow.

    During your testing, monitor the following:

    • Java process memory usage

      If the process reached the maximum configured for java memory and the performance is still unacceptable, consider increasing the memory allocation. Stay within the guidelines outlined in Tuning Java memory allocation. Restart the secondary server every time you update the jvmsettings.cfg file.

    • System memory usage
    • CPU usage
    • Resource swapping
    • Java garbage collection

      If you see a lot of garbage collection threads, enable logging for garbage collection. Update jvmsettings.cfg to add these parameters:

      • -verbose:gc
      • -XX:+PrintGCDetails
      • -XX:+PrintGCTimeStamps
      • -XX:+PrintGCDateStamps
      • -Xloggc:/aiw/aiw1/trace/gc_secondary.log
      • -XX:+UseGCLogFileRotation
      • -XX:NumberOfGCLogFiles=10

        Adjust this setting to meet your needs.

      • -XX:GCLogFileSize=8m

        Adjust this setting to meet your needs.

      You must restart the RICOH ProcessDirector service every time you update jvmsettings.cfg.

      Review the logs to determine the frequency and duration of full garbage collection. If the process takes more than five to ten seconds to complete, it could affect performance.
    • I/O use

  11. After each test, adjust settings to optimize processing. When you finish adjusting the step, update your production workflows to use it instead of the current BuildPDFFromDocuments step.
Parent topic: Consideraciones de capacidad y rendimiento para las características de procesamiento de documentos