App Store Product Page Optimization: How to A/B Test Your Screenshots and Icon

Product page optimization is the A/B testing feature built into App Store Connect. You create alternate versions of your product page, which Apple calls treatments, and each one can swap in a different app icon, a different set of screenshots, or a different app preview video. Apple then shows each treatment to a percentage of randomly selected App Store visitors and compares how often they download against your original page. It costs nothing and the traffic is real App Store traffic, not a panel or a survey.

Apple's rules set the shape of every test. You can test up to three treatments against the original. Treatments are shown to people on the App Store on iOS 15 and iPadOS 15 or later, and a person who lands in a treatment sees that same treatment whenever they visit during the test. Each treatment can be localized into all the languages your app supports or only the ones you pick. A test runs for 90 days or until you stop it, you can only run one test at a time, and you cannot change a test once it has started. Tests are not available for custom product pages or for product pages on the App Store for Apple Watch or iMessage.

Not before launch. App Store Connect Help says you can configure a test if your app has a Ready for Distribution or Pre-Order Ready for Distribution status and is live on the App Store. So the first version of your listing is always a judgement call, which is why the groundwork in [App Store screenshots that convert](/blog/app-store-screenshots-that-convert) and [App Store Optimization for a brand-new app](/blog/app-store-optimization) matters more than any test you run later.

The harder limit for a small app is traffic. Apple's test setup includes an optional Estimate Your Test Duration step: you choose the improvement in conversion rate you hope to find, and App Store Connect uses your own daily impressions and new downloads to estimate how long the test needs and how many impressions it takes to reach an outcome with at least 90% confidence. Apple is direct that it may not be possible to reach your target within 90 days. If the estimate comes back longer than that, Apple's suggestion is to cut the number of treatments or raise the traffic proportion.

My honest view for most members: if your app gets a trickle of impressions a day, run one treatment, not three, and test the change you think is biggest. A three-way icon test on a page almost nobody sees just runs out the 90-day clock. Get the traffic moving first with [your first 100 users](/blog/how-to-get-your-first-100-app-users) or a small [Apple Search Ads campaign](/blog/apple-search-ads-for-indie-apps), then test.

Inside App Store Launch Club I call this the One-Change Rule: each treatment changes exactly one thing against the original. A new first screenshot, or a new icon color, or a preview video added in front. Never all three at once. Apple's own guidance points the same way, advising you to consider how many elements you change in a treatment so you can tell which one led to a result. If Treatment A has a new icon, new captions and a new screenshot order and it wins, you have learned that one of those three helped and possibly another one hurt.

The order I test in follows what a visitor actually sees. In search results the icon and the first screenshots do almost all the work, so that is where the first test goes. Good first tests for an indie app:

Write the hypothesis down before you start, in one sentence: what you changed and what you expect. Apple asks for a reference name of up to 64 characters, and I put the hypothesis straight into it so the result in App Analytics reads like an answer to a question.

You need the Account Holder, Admin, App Manager or Marketing role. These are the steps from App Store Connect Help:

One timing detail: Apple warns that releasing a new app version while a test is running may affect the results if that version contains the assets or metadata being tested. If you have an update ready, either ship it before the test starts or keep its screenshots and icon untouched until the test ends. The routine for the update itself is in [how to submit an app update](/blog/how-to-submit-an-app-update-to-the-app-store).

Screenshots and previews are uploaded straight into App Store Connect. Icons are different. App Store Connect Help says any icon you want to test must be part of the app binary for the current App Store version, built with Xcode 13 or later with support for alternate icons in asset catalogs, and sized to 1024x1024 pixels. The App Icon tab in the treatment only appears when the binary already contains more than one icon. When someone downloads from a treatment with an alternate icon, that icon shows throughout the download and on their device.

For an Expo app, that makes an icon test a native change. Expo's documentation on EAS Update lists anything that changes native code, or anything that requires a new app binary, as work for EAS Build rather than an over-the-air update. So [pushing an update without App Store review](/blog/how-to-push-app-updates-without-app-store-review) cannot add an alternate icon. The sequence is: add the icons to the iOS build, make a new production build, get that version approved and live, then create the test.

If you are still settling on the original icon, [how to make an app icon for the App Store](/blog/how-to-make-an-app-icon-for-the-app-store) covers the design side. For screenshot treatments, [automating App Store screenshots with Claude Code](/blog/how-to-automate-app-store-screenshots-with-claude-code) makes producing a variant set a small job instead of an afternoon.

Results live in App Analytics, not on the Product Page Optimization screen. Apple says results appear after there are five first-time downloads associated with the test. The dashboard shows whether the test needs more data or whether a treatment is performing better or worse than the baseline, with impressions, conversion rate, percent improvement and confidence for each treatment. The baseline is your original page by default, and you can switch it to compare treatments against each other.

A 90% confidence threshold still leaves room to be wrong, and a small percent improvement on a small app is a small number of extra downloads. I treat a winner as the new baseline and the start of the next test, not as proof. Apple's own published results are a useful calibration: the developer stories on its product page optimization page show single-digit percentage lifts, including one where the tester's hypothesis lost to the original. A treatment losing is a result too. It tells you what your users do not respond to.

Keep a one-line log of every test: the hypothesis, the dates, the confidence and the outcome. After three or four tests that log is the most useful ASO document you own, because it is built from your own users and nobody else can copy it. Carry what you learn into [your App Store description](/blog/how-to-write-an-app-store-description) and your [keyword research](/blog/how-to-find-app-store-keywords), since a message that wins in screenshots usually belongs in the text as well.

App Store product page optimization is Apple's free A/B test for your icon, screenshots and app previews. You need a live app in Ready for Distribution, you can test up to three treatments against the original, one test at a time, for up to 90 days, on visitors running iOS 15 or iPadOS 15 or later, and the test cannot be edited once it starts. New screenshots and previews need App Review but not a new app version; alternate icons must already be in the live binary, which for an Expo app means a new EAS build first. Results show in App Analytics after five first-time downloads, Apple recommends waiting for 90% confidence before acting, and applying a winner is permanent and covers screenshots and previews only. Follow the One-Change Rule and use the duration estimate honestly: if it says more than 90 days, test fewer things.

The One-Change Rule came from watching members change the icon, the captions and the screenshot order in one treatment, win, and have no idea why. Inside App Store Launch Club you can post your current product page and your test plan and get a second pair of eyes on which single change to test first, before you spend 90 days finding out. Join at applaunchclub.co for $9 a month and bring your listing.