With brands competing on the basis of customer experience, across an increasing number of devices and channels, it is now essential to ensure images and media are fully optimised for every visitor.
While designs target a specific set of mobile and desktop resolutions, there are now so many devices in use that it is no longer feasible to design for every scenario. Image sizes and aspect ratios that work well on one device, do not necessarily work well on another.
As brands look to create content quicker, editors no longer have the time to create multiple crops. A better solution is needed.
Handling 1,400 screen resolutions
From smartphones, to tablets, to smart TVs, the number of devices has exploded over the last few years. While working on improving the customer experience for Sassoon Academy, we found that their visitors were using over 1400 unique screen resolutions.
Supporting multiple aspect ratios for London Live
Back in 2014, we experienced a similar challenge. Building the digital platform for London Live, a new TV station for London, we needed to set up the content, media and image workflows for this busy news channel.
The editorial and publication workflows for the new TV channel would need to handle a considerable volume of images, many of which focused on individuals, groups and faces. We needed to pay careful attention to how these images would be rendered across a range of aspect ratios, from mobile phones, to tablets and TV screens.
On desktop devices, we decided on using three standard image ratios: 16x9 for video covers; 3x2 for episode carousels; and 5x2 for hero images used across the top of the screen. However, on mobile devices these looked far too skinny, and we decided to render video covers at 14x9 and episode carousels at 1x1.
This meant that the same content would appear differently across desktop and mobile. While this is not a problem if the image aspect ratio remains the same (developers can handle this with ease), in our case, the images also changed in dimension and aspect ratio.
The problem with automatic crops
One basic solution is to use HTML and CSS hacks for automatic crops (such as “background-size: cover”) to render the images inside many different shaped containers.
However, this approach doesn’t take into consideration the contents of the image. The default is that these automatic crops will be focused on the information in the center of the image, meaning that the context of the image is easily lost.
While uploading multiple image crops might seem like the obvious way forward, for today’s visually rich experiences this would create a considerable amount of work for content editors - and is time that should be better spent elsewhere.
We wanted to find a service that would identify the most important points of an image and focus on them when cropping. This would ensure the context of the image was still conveyed when the aspect ratio changed, such as making a wide banner into a square image, or vice versa.
The ideal crop would be as follows:
Device-specific aspect ratios for Sassoon Academy
With new design work for Sassoon Academy this became even trickier, as the composition of original photos were unbalanced by design, with the subject of the image being either positioned to the left or right.
An automatic approach to cropping these images resulted in the focus of the image being lost entirely.
A smarter solution
Back in 2001, two computer scientists, Paul Viola and Michael Jones triggered a revolution in the field of computer face detection. Their breakthrough was an algorithm that could spot faces in an image in real time.
They focused on faces when viewed from the front, ignoring any seen from an angle. Given these bounds, they realised that the bridge of the nose usually formed a vertical line that was brighter than the eyes and they also noticed that the eyes were often in shadow and formed a darker area.
Now used by companies like Facebook and incorporated in cameras, the software uses these algorithms to scan an image to identify edges, regions with skin colour tones and regions high in saturation.
When deciding how to crop an image, the software generates a set of candidate crops, ranks them to focus on detail in the center of the image and returns the crop with the highest rank.
Let’s assume we want a square crop and are starting with the following image:
The software sees the same image as this:
And returns an square crop as follows.
Let’s take a more complex example. Starting with this image:
The software sees the image as this:
and returns this crop:
How to implement this technology
This advanced image processing technology is no longer exclusive. There are two main approaches that companies can use to successfully include automated and intelligent image processing.
For brands wishing to quickly improve the image presentation, a new group of image delivery services has sprung up that can be integrated very easily to address these issues.
Offering intelligent image cropping, they examine the actual content of the image, determine the correct focus area and ensure that the image context is not lost, regardless of the size and shape of the image being shown
There are two providers that we recommend, having used both with great success.
Cloudinary offers a comprehensive, cloud-based image management solution and has been around since 2011.
For teams building mobile applications, working with a high volume of user generated content or running their own technology stacks, an open-source version of this software can be installed as a Node.JS package.
Using these technologies, it is now possible to deliver excellent image presentation to all devices, with one single source image. The image is rendered in the ideal format and at the required size. All visitors see perfectly cropped images, regardless of device, reducing network traffic and improving site load times.