Two tools can disagree because they examined different bytes, selected different records, applied different transformations, or attached different meanings to the same value. Choosing the result from the more familiar interface does not explain the conflict. Work from the smallest reproducible discrepancy and preserve both original outputs.
Freeze the comparison
Identify the input by manifest and hash, then record each product's version, plugin, settings, platform, and operation. Preserve the reports and logs before changing settings. State the exact difference: a missing record, count mismatch, time conversion, decoded value, or inference. A large report-level comparison is easier to misunderstand than a single record with an addressable location.
Check that the inputs are equivalent
Compare acquisition scope, included companion files, source snapshots, encryption state, and export formats. If one tool consumes a native database and another consumes a spreadsheet prepared from it, the comparison includes an extra transformation. Locate where that transformation occurred. Two tools operating on different snapshots cannot establish a parser disagreement until the relevant input difference is accounted for.
Separate extraction from presentation
Retain raw values alongside rendered values. Investigate filters, duplicate handling, deleted-record inclusion, time-zone settings, and default field normalization as applicable. Ask whether a blank field means no value, unsupported decoding, redaction, or a suppressed display. Do not assume a shared column label implies the same underlying semantics. Document the product's meaning where its original documentation provides one.
Establish a reference outside the vote
Trace the field to its source bytes or a documented native representation when feasible. Build a small authorized test input containing the feature in dispute and record its construction. Use another method only after checking its dependencies. A third interface sharing a parser with one of the first two adds less independent support than its different branding suggests. Agreement by majority is not ground truth.
Classify the outcome
Use clear categories: different scope, different supported semantics, confirmed defect under stated conditions, or unresolved. Record the test that distinguishes the categories. A vendor response is useful evidence about implementation, but preserve the supplied example and verify any fix before depending on it. If a correction affects earlier analysis, identify the conclusions and derivatives that require review.
Report a bounded finding
State the input, versions, disputed field, observed outputs, reference basis, and remaining uncertainty. “Version A omitted this record variant under these settings” is more informative than “Tool A is unreliable.” Keep unrelated product judgments out of the finding. If source restrictions prevent publishing the example, describe the limitation and preserve a protected reproduction package for authorized review.
- Build an independent reference
Keep expected values separate from the method under test.
- Preserve the reproduction
Package the decisive input and settings with the observed result.