Testing Algorithm Changes on Real Orders Before Publishing

Compare the live algorithm (A) with a draft or previous version (B) on single or batch orders in the Simulator, view the selected source and reason; step-by-step instructions and video.

The Simulator tests an algorithm change on real orders before publishing and compares it side-by-side with the live version. On the OMS – Order Management › Simulator screen, you simulate either a single order or multiple orders together. This page provides step-by-step instructions; a video of the same explanation is below.

<video controls preload="metadata" playsinline width="1920" height="1080" poster="https://files.docs.hebiar.com/videos/console/oms/simulator/simulator.en.jpg" src="https://files.docs.hebiar.com/videos/console/oms/simulator/simulator.en.mp4"><a href="https://files.docs.hebiar.com/videos/console/oms/simulator/simulator.en.mp4">Open the video</a></video>

For all screen areas, see Simulator. Draft preparation is explained in Algorithms: weights, multipliers, and drafts, and how rules apply to orders is described on the Matching rules page.

Before You Start

  • A is always the live algorithm (e.g., Live · Default algorithm v14).
  • B is the side you want to compare. It can be one of:
    • a saved draft of an algorithm,
    • an old Version,
    • Saved city rules (not published),
    • Edit by hand (not saved): a draft edited only for this comparison and not saved.
  • The Simulator evaluates the order according to matching rules: the algorithm of the rule that matches the order runs. For example, an order going to Istanbul is calculated using the formula of the live city rule for Istanbul if it exists. So you test not a single algorithm but the algorithm of the rule that fits the order.
  • Only open orders are simulated. Invoiced, shipped, canceled, or payment-pending orders return empty.
  • Draft comparison requires OwnerAdmin or TenantAdmin role; if you lack permission, only live results are shown.
  • The Simulator does not publish anything or change the order.

Finding the Order to Simulate

  1. Open the Orders screen.
  2. Select Last 30 days and click Filter.
  3. Choose an order with status Created, meaning it is not yet invoiced or shipped, and note its number.

The Simulator also suggests recent open order numbers under the Recent open orders heading; tapping one starts the simulation immediately.

Comparing a Single Order

  1. On the Simulator screen, be on the Single order tab.
  2. Select the side to compare from the B list (e.g., a previous version).
  3. Enter the order number in the Order number box and click Simulate.
  4. The result table shows each row as a source and carrier option that can fulfill the order. Scores for A and B are side by side; the lower score is chosen and marked with Chosen.
  5. The bar colors indicate where the score comes from: Preparation, Carrier, Stock effect, Workload effect, and Other (order formula). Scores of the two algorithms are not compared against each other; each is ranked internally.
  6. The Why? section below explains the choices of both sides and the closest competitor. The summary line states the result; for example, "The draft removes this order from ... source instead of .... Preparation + carrier cost decreases by ...."
  7. When B is a draft or old version, results are shown with the Estimated label: the best 100 plans are rescored by the draft formula compared to the live formula.

Batch Comparison

  1. Switch to the Batch comparison tab. The B side selection is retained.
  2. Use Paste a list to paste order numbers separated by spaces, commas, or lines; or fetch recent open orders by date, channel, and delivery city with Recent open orders.
  3. Click Compare.
  4. Summary cards show how many orders changed source (Source changed), how preparation and carrier costs changed, the share leaving the store, and average carrier distance. Below, workload per carrier region and source is listed.

Checking the Result

  • Drafts are not published in the simulator. Publishing is done by re-comparing sample orders on the Algorithms screen; the Open in Algorithms button in the result box takes you there.
  • If you tested a city rule change, you publish it from the Matching rules screen.

Common Situations

  • Symptom: Simulation returned empty. The order is invoiced, shipped, canceled, or awaiting payment. Select an open order.
  • Symptom: A and B choose the same source. The change does not affect ranking for this order. Try orders going to different cities or channels, or use batch comparison.
  • Symptom: I selected a draft but only live results appear. Draft comparison requires admin role or the draft has not passed validation; read the warning on screen.
  • Symptom: I changed the algorithm but the result was not calculated with the expected algorithm. The Simulator uses the algorithm of the matching rule for the order; check the order’s delivery city and rules.