Print comparison test details
The test
Over a three-month period ending November, 2003, we sent 26 test shots repeatedly to 10 print providers.
The test shots were picked from a cross-section of prints our customers have made, each chosen for its challenge to the printing process. They range from professional quality to under-exposed shots in bad light from consumer cameras.
The testers ranged from a renowned printing expert, to pro photographers, to consumers with point-and-shoot cameras. Everyone viewed the prints under household tungsten lights or daylight, some both ways to see how it influenced their decisions.
We did not test under fluorescent light, because we felt most people would open their print packages and make their judgements under either household tungsten lighting or diffused daylight.
The final four
To make it manageable, we showed the prints first to a group of 16 customers, who quickly showed strong preference for 4 sets of prints. They came from Shutterfly, Ofoto, and two sets came from a lab in Georgia that caters to pro photographers by the name of EZprints. The two EZprints orders were processed with different options, explained below.
Customers also reacted favorably to prints from PhotoAccess, SnapFish, and dotPhoto, but very negatively to approximately 6 of WalMart's prints.
We then placed all 4 copies of each print from the final four on its own white board and conducted a blind test with an expanded group of 30 customers, one at a time. They were not told which company produced which print, or even who the final four were.
We asked each customer to choose their favorite and least favorite for each print. We asked them why they chose the prints they did and we made note of their own skin tone, which may have influenced their tastes.
Scan of prints from Shutterfly (top left), Ofoto (top right), WalMart (lower left) and EZprints (lower right).Observations
Most testers were surprised to see how much some shots varied, and how they liked added yellow (Shutterfly) in some cases and not in others. They would frequently comment that they believed shot A was more accurate, but they had to say that "in this case" they preferred shot B because it was more flattering.
Most of them commented about how often it came down to personal taste and how they felt inconsistent with their criteria.
By and large, however, they expressed their sentiment that they hoped the most accurate color rendition would win the day. And, taken as an average across all prints, it did.