Controlling Decimal Precision in Reports
How to control decimal precision in QC-CALC 4.1 report templates
Print Friendly View
written: 11/26/2023
last modified: 04/10/2024

This White Paper explains how to control the formatting of numbers within a report in general and gives step-by-step directions for changing the formatting for a First Article report.

General Number Formatting

When QC-CALC sends numbers to the reports those numbers have the full 7 digits of precision included. In most cases, you do not want to see that much precision and the templates were set up by default to follow the precision you chose for specific dimensions. This section is going to explain formatting a hard-coded 4 decimal places number, a number without a decimal point at all, and a number that will use the number of decimal places defined for the specific Dimension being processed at the time.

The function that is used to format numbers within the Report Designer is “FStr$([value],[format to follow])”. All examples below will use this function, but the “format to follow” will be different for each.

In each example there will be 7 places to the left of the decimal point reserved. If you do not reserve enough space to the left of the decimal to hold your number you will see a “********” in your report. Add more spaces to the left and that should fix that issue. The example below shows the “Actual” being sent and 7 places to the left of the decimal and none to the right being reserved to show that number.

FStr$(Actual,"------&.")

The following items are commonly used in the FStr$ line:

-

Digit or sign, if negative

+

Digit or sign

#

Digit or space prefix

&

Digit or '0'

.

Decimal point

If just “#######.####” is used as the formatting instead of using the – signs and the & for the placeholder next to the decimal point, a couple of different issues could happen:

  1. If a negative number is encountered, the negative sign will not be displayed. For example, “-1.3456” would end up displayed as “1.3456”, which is very wrong.
  2. If a number between 0 and 1 is encountered it will be displayed as “.1234” instead of the “0.1234” you would probably prefer.

4 Decimal Places

For this example we’re going to set the format function to always show 4 decimal places to the right of the decimal point along with the 7 places to the left.

FStr$(Actual,"------&.####") -> 8.1234

No Decimal Places

For this example we’re going to set the format function to always not show any numbers to the right or a decimal point.

FStr$(Actual,"------&") -> 8

Dynamic Decimal Places

For this example we’re going to set the format function to show the number of decimal places set within the QC-CALC and passed in to the template in the “Precision” variable in this case.

FStr$(Actual,"------&."+Rep$("#",Precision)) -> 8.12345 (with Precision = 5)

For this example, we are using the “Rep$” (repeat) function to repeat the # character the number of times specified by the precision. This means if your decimal precision was set to 5 in QC-CALC, then the formatting line would be equivalent to:

FStr$(Actual,"------&.#####")

First Article Example

The directions shown here will walk you through changing the formatting of the numbers in the Nominal column in the data area of the table of a First Article report.

  1. From QC-CALC Real-Time choose the Report > Design Template menu as shown to the right.

  1. Choose the OnePiece.FST template from the list of files that appears and click Open.
  1. The Report Designer now appears on the screen.

4. Use the Objects area in the upper left area of the screen and double-click on the “Table 1” item.

  1. in the Table Contents window that appears, choose the Data Line tab at the top and then double-click the “Nominal” line in the Line Definition 1 area.

  1. In the Edit Table window that appears, you will see the FStr$ as the 2nd half of the line in the bottom section of the window. Currently the Dynamic formatting method is being used, but we’re going to switch it to a hard-coded 5 decimal places instead.

  1. Remove the +Rep$(“#”,Precision) section at the end of the line, but make sure you leave 2 “)” at the end.
  1. Now, to hard-code the 5 decimal places, type ##### between the . and the .
  1. The line you end up with should be:

  1. Click OK in the Edit Table and Table Contents windows, then choose File > Save to save your changes.
  1. Now when you run that report you should always see 5 decimal places for the Nominal regardless of the precision set for each dimension.