How to Run a Credible SuiteCommerce Performance Audit
A large audit is only useful when readers can tell what was measured. A store count and a Lighthouse median are not enough. The sample, test date, page selection, runner settings, exclusions, and analysis code all affect the result.
This article previously reported results for 286 stores without enough supporting methodology in the repository. Those numbers have been removed until the dataset and process can be verified without exposing merchant information.
Define the population
State how stores were found and what qualified as SuiteCommerce. Record:
- collection dates;
- countries or markets included;
- whether inactive, redirected, password-protected, or duplicate domains were excluded;
- how platform detection was confirmed;
- how many candidates failed and why.
Do not treat a JavaScript namespace as conclusive version evidence. Themes, cached assets, extensions, and old files can outlive an application release. If version cannot be confirmed, label it unknown.
Keep denominators visible
If 286 domains were collected but only 118 produced valid measurements, every percentage must identify whether its denominator is 286, 118, or another subset. Report missing data instead of silently dropping it.
A useful flow table is:
| Stage | Count | Exclusion reason |
|---|---|---|
| Candidate domains | ||
| SuiteCommerce confirmed | ||
| Eligible pages found | ||
| Successful audit runs | ||
| Included in final analysis |
Make tests repeatable
Publish the non-sensitive parts of the method:
- Lighthouse and Chrome versions;
- hardware or runner class;
- network and CPU settings;
- geographic test location;
- cold or warm cache policy;
- number of runs per page;
- how home, category, and product pages were selected;
- how bot blocks, consent screens, and transient failures were handled;
- aggregation method and rounding.
Lighthouse produces lab data under simulated conditions. It is not real-user data. Google's Core Web Vitals documentation explains the difference and uses field measurements at the 75th percentile for the Core Web Vitals assessment.
Protect merchants
Store domains, account identifiers, page URLs, screenshots, and raw traces may reveal commercial or customer information. Keep raw data access restricted. Publish aggregate results only after checking that small groups cannot identify a merchant. Obtain permission before naming a store or partner.
Avoid unsupported conclusions
A Lighthouse result can identify likely page-performance work. It cannot prove lost revenue, agency quality, or search impact for a store. Correlation between a visible extension and a score does not establish causation.
Report medians and distributions with confidence intervals where the sampling method supports them. Keep observations separate from explanations that still need testing.
Publication checklist
- Dataset provenance recorded
- Collection and audit dates recorded
- Platform and version detection validated
- Denominators shown for every result
- Failures and exclusions reported
- Runner configuration preserved
- Analysis can be reproduced
- Merchant data reviewed for privacy
- Lab results are not called field data
- Business outcomes are not inferred from speed alone
Once those checks are complete, aggregate findings can be restored with a link to the method and an audit date.
