What a good first month looks like.

Any new system in a store, whether it is a helpdesk, an inventory tool or an assistant that drafts catalogue changes, gets judged on a feeling by the end of its first month. It seemed to help, or it did not. A feeling is a poor basis for a decision you will live with for a year. This post sets out a way to run the first 30 days so that at the end you have numbers, and a clear answer on whether to keep going, change how you use it, or stop.

The shape is simple: measure before you change anything, start by only reading, make small reversible changes with a person checking them, and compare against your own baseline at the end. Each step comes with the evidence for it, and where the evidence is thin we say so.

Before day one: write down where you are

The strongest case for starting with a baseline comes from a place far from ecommerce. In 2009 a team from the Harvard School of Public Health and the World Health Organization published a study of a 19 item surgical safety checklist at eight hospitals in cities from Seattle to Manila. They studied 7,688 patients, 3,733 before the checklist was introduced and 3,955 after, between October 2007 and September 2008. Major complications fell from 11% to 7%, and inpatient deaths from 1.5% to 0.8%. The study could only say that because it measured each hospital before the change. Without the first group, the second would have been a set of numbers with nothing to compare against.

The comparison was before and after, not a controlled trial, so it cannot rule out other changes over the year. That caveat applies to your store too, and it is the reason to write the baseline down and to date it.

For a Shopify store, the baseline has two parts. The first is sales measures. Shopify defines average order value as total revenue divided by the number of orders, and conversion rate as orders divided by visits, multiplied by 100. Its guide warns that an average can mislead: one statistician quoted there says no measure of central tendency is best, but using only one is certainly the worst. In its worked example, a store with a mean order of $24 had a most common order of only $15, so record the median and the most common order value next to the mean.

Shopify's conversion guide cites Dynamic Yield figures of a 2.66% global average conversion rate across more than 400 brands, with desktop at 3.7% and mobile at 2%. It adds that rates depend on what you sell, the price, how often people buy and where traffic comes from, and recommends tracking by device, traffic source and product, not as one blended number. Public averages are useful for scale only. Your own last 30 days, split the same way, is the baseline that matters.

• 11% to 7%

  major complications before and after a surgical checklist in the 2009 eight hospital study

  Harvard Gazette report on the WHO study, 7,688 patients

• 6.3 hours

  all industry median first response time for ecommerce brands in the $10M GMV band, with a 5.5 times spread by vertical

  Gorgias Ecom Lab, March 2026

• 2.66%

  global average ecommerce conversion rate, from more than 400 brands

  Dynamic Yield, cited by Shopify

The second part is service measures, if you use a helpdesk. Shopify's customer service guide recommends tracking customer satisfaction, customer effort, first response time, the ratio of support requests to orders, and the types of conversation. It warns that averages hide individual bad experiences and that comments should be read alongside scores. It also cites that 80% of customers now expect a reply within 24 hours and 37% within an hour.

Gorgias, a helpdesk vendor, analysed its own merchants' data in March 2026, using accounts with at least 30 tickets, in the $10M gross merchandise value band. First response time varied 5.5 times across 14 verticals, from 1.6 hours in hardware to 9.1 hours in apparel, with an all industry median of 6.3 hours. Ticket volume ranged from about 20 to about 46 per 100 orders, and customer satisfaction varied by only 0.2 of a point. Its advice is to find your own row in the table, because comparing with the wrong group is misleading. Gorgias also publishes general targets in a separate benchmarks article: first response by email under 24 hours, with under 12 hours called excellent, and customer satisfaction of 80% to 85% as the standard benchmark. Treat those as rough guides to check your own numbers against, not as goals. For a small UK store, the lesson is to compare yourself with yourself first, and with similar stores only second.

Write all of it on one page with the date: 30 day revenue, orders, average and median order value, conversion by device, first response time, tickets per 100 orders, and a count of products with missing or poor titles, descriptions or tags. The last one is a number you can produce in an afternoon from an export, and it is often the most revealing.

Week one: read only

The first week should change nothing in your store. The aim is to see what the system sees and check it against what you know. If it reads your catalogue, ask it questions you already know the answers to: how many products have no tags, which items are out of stock, what customers asked about this week. Where it is wrong, you have learned how far to trust it. Where it is right, you have learned what it can do without any risk.

Do three things in these days. Confirm what it can reach, and what it cannot, because a tool with wider access than the job needs is a risk with no benefit. Decide who approves, by name, and who covers when they are away. And set the review times, so that approvals do not wait on whoever happens to check.

Keep the first week small. A few tasks that all get done are worth more than a long list that does not, and it leaves you time to run the shop. The Harvard report on the surgical study makes a related point: the checklist was a single page that took only minutes to complete, at three moments in an operation. A short routine that people actually use beat a long one that nobody would.

Week two: a few small changes

Now let the system propose changes, and keep them narrow. Pick one kind of change, such as product descriptions in a single collection, and a small number of products, 20 or so. Narrow means that when something is wrong you can see the pattern, and it means you can undo it. Look at each draft before approving. Decline those you disagree with, and write down why.

This is where a person checking the work can fail, and the evidence on it is worth knowing. Goddard, Roudsari and Wyatt reviewed 74 studies on automation bias, the tendency to over rely on automated advice, mostly in healthcare. A meta analysis of four studies found that erroneous advice was more likely to be followed when it came from a decision support system, with a risk ratio of 1.26, meaning that when the system was wrong it raised the risk of an incorrect decision by 26%. Trust that is not matched to how reliable the system is was described as possibly the strongest driver of over reliance, and increased workload made users more likely to overuse the advice.

Those are clinical settings, and a catalogue edit is a lower stakes decision. Even so, the mechanisms carry over. The same review found that training reduced these errors, that making people accountable for their decisions may prevent them, and that showing information instead of a single recommendation reduced over reliance. Three habits follow. Read the reasoning, not only the result. Keep the batch small enough that you are not rushing, which also answers the workload finding. And be named on each approval, so it is a decision somebody owns.

A simple way to test your own checking is to put a deliberate mistake into one batch, such as a wrong product type in a description you wrote yourself, and see whether it is caught. It is a rough test, not a study, but it tells you early whether approvals are real.

Weeks three and four: make it routine and count

If week two went well, widen the scope a step at a time: another collection, another type of change, or helpdesk replies on one topic such as delivery status. Each time you widen, keep the old rule of narrow, reviewed and reversible. This is also when the daily rhythm forms. Set a fixed time to approve, so it takes ten minutes and does not interrupt other work.

Keep a short tally during these two weeks. For each batch, note how many drafts were approved as written, how many were edited, how many were declined, and the reasons. A high approval rate with few edits is a good sign only if you have checked that the reviewing was real. A high decline rate is useful information about where the system is weak, and it tells you which job to keep for a human.

Watch for the system making work, not removing it. If approving takes longer than doing the job yourself, or if you spend the saved time correcting repeated errors, then it is not helping on that task yet, however good it looks. That is a result, and you should record it as one.

Day 30: compare with the baseline

Return to the one page you wrote before day one. Pull the same measures over the same length of window and put them side by side. Be careful about what the numbers can tell you in a month. Sales and conversion move with season, traffic mix and promotions, so a change in either is weak evidence about a tool that edits descriptions. The measures closest to the work are the better test: time spent on the task, tickets answered, first response time, products fixed, and the share of drafts you approved without edits.

Read the words as well as the numbers. Shopify's guide cautions that most satisfaction tools show an average score that can hide individual poor experiences, and that customer comments explain why people were happy or not. If a drafted reply was approved and then a customer wrote back unhappy, that thread tells you more about the first month than any average. Pull out the five worst and the five best, and read them in full. The same goes for order value: if the mean moved, check whether the median and the most common order did, since one large order can shift the mean alone.

The standard to apply is the one a store owner would apply to anything else: is it better than before, and by enough to matter? If the measures closest to the work improved, the sales measures are flat, and your approval tally shows real review, that is a reasonable result for a first month. If nothing moved, or it got worse, say so and decide whether to change the scope, the process or the tool. A month is long enough to know whether to carry on, and too short to claim a sales effect.

Where BYOM fits

Connect your store and, if you use one, your helpdesk. Kina starts reading straight away. Each change arrives with its context. Approve the ones you agree with and decline the rest with a reason.