DxO PhotoLab 9 AI masking tools explained, with examples

DxO PhotoLab already had a formidable set of local adjustment tools, including some that rivals don’t have. PhotoLab’s clever control point masks and control lines, which effectively combine linear masks with selective masking, are unique to DxO. But other makers had already introduced AI masking with automatic region, subject and object detection, so it was about time PhotoLab caught up – and that’s exactly what DxO has done with the launch of PhotoLab 9.

• Read this full DxO PhotoLab 9 review

The AI masking tools are not the only new feature in PhotoLab 9, but they are perhaps the most important. They don’t replace any of the existing local adjustment tools but work alongside them. Better still,  it’s possible to use them together with DxO’s new system of masks and ‘submasks’.

The new AI mask tool is the first in the row of tools at the top of the Local Adjustments panel. When you click on this you’ll see a strip of AI masking tools appear under the image – they’re not in the Local Adjustments panel itself. These AI tools come in three modes:

Option 1: Area selection

The first two buttons are for selecting and removing areas. With the ‘add’ button selected you just  move the mouse pointer over the image and it will highlight different areas with a red overlay – you just click to confirm the selection and create the mask, which then appears in the Local Adjustments panel. 

If the mask doesn’t include all the objects you need it to, then you can hold down the [shift] key modifier to add new areas to the mask –  these will appear as new ‘submasks’ in the Local Adjustments panel. If you don’t hold down the shift key, then the tool will create a whole new mask instead of adding to the existing one.

If the mask includes some areas you didn’t want it too you can swap to the ‘remove’ button and click on these to remove them from the mask. This is an effective way to remove see-through parts of an object, for example, or nearby objects that you want to separate from your main subject. Alternatively you can hold down the alt/option modifier and click to subtract rather than add areas.

Option 2: Object selection by marquee

The area selection tools are pretty effective, but there’s another option that could prove more useful where you have a clearly-defined object that you want to mask. Here, you can swap to the next pair of buttons for adding and removing objects. This time, you don’t see a red mask overlay as you move the mouse pointer around the image. What you actually have is a rectangular marquee tool which you drag over the object you want to mask, making sure you include it all. The tool will then look for and select the main object within this area – and the tighter you can wrap the marquee around the object, the more accurate the mask is likely to be.

This tool is really effective where you want to mask a definite object rather than a broad area of tone. You can add additional areas with the ‘add’ tool or remove parts of the object with the ‘remove’ button. Again, make sure you add a submask first before you add an object, but you don’t need to do this with the remove tool – see my note above.

I like the object masking tools a lot – it seems to work really well. But there’s a third option on this horizontal tool strip – a pre-defined subject type drop down.

Option 3: Named subject types

With this tool, PhotoLab will attempt to identify and mask any of the following options in the scene: Sky, Subject, Background, People, Animals, Flowers, Vehicles, Hair, Faces, Clothes.

I haven’t tried this on every subject type with lots of different images, but in my tests so far this seems to work very well. It successfully identified vehicles, including motorcycles (even when there was more than one in the frame), animals and skies.

Mostly the masks are pretty good. As with all other AI masking tools I’ve tried, sometimes there’s a little ‘overspill’ and sometimes there’s a hard edge where there’s a soft boundary in the image, and as usual the AI can’t cope with ‘holes’ in objects, like windows, or the spaces between wheel spokes. However, it is possible to fix this with ’submasks’…

Combining masks with submasks

DxO’s new system of masks and submasks means that you can use more than one masking tool to build your mask. For example, you might use an AI object mask to select a foreground object to lighten it. This creates a new mask with the object mask as a ‘submask’ within it. To add to the mask, you just click the ‘ new submask’ button, choose your masking tool and create the additional submask. For example, you might use the linear gradient mask tool to drag up from the base of the image to the object you’re enhancing, so that if you lighten the object you lighten the foreground at the same time in a natural looking way.

Can you subtract or ‘intersect’ masks?

This is a very useful feature in Adobe Lightroom Classic, for example, and one I’ve used a lot. If you use an AI mask to select the sky in an outdoor scene, for example, you get a very hard selection right up to the horizon line, so that if you apply strong adjustments to darken a bright sky, you get a very unnatural cut-off at the horizon. In reality, the sky adjustment should fade more gradually towards the horizon line.

Now you might argue that a straightforward linear gradient will work for this, but that doesn’t work if there are tall objects jutting up into the sky like trees, buildings or mountains which would also be darkened. This is where you need AI sky selection – to exclude these objects from the mask.

In Lightroom Classic you can blend one mask with another this by ‘intersecting’ these two masks so that only the areas where both masks are active will make up the final mask for your adjustments. PhotoLab 9 does not have an ‘intersect’ option, but it does have a workaround.

If I want to blend in a darkened sky more naturally towards the horizon, you can start off with an AI sky mask, but then you need to use the ‘add submask button’ in the Local Adjustments panel and then use the linear gradient tool to drag upwards into the sky from the horizon line.

At this point you won’t see much difference because the new linear gradient mask is on top of the original sky mask – they cover the same area. But if you now use the ‘Invert Shape’ button at the bottom of the panel with the linear mask still selected, it will now be ‘subtracted’ from the sky mask, and it will fade it in more gradually from the horizon line.

That might sound almost unfathomable when it’s written down, but these screenshots should make it clearer.

The new AI masking and mask combining tools in PhotoLab 9 are very powerful but you might need to spend a little time with them before they become second nature. This is a major new feature in this version of PhotoLab, though, and DxO does seem to have got everything right at its first attempt. Bravo!

• DxO PhotoLab 9 review
• More DxO PhotoLab articles
• DxO Nik Collection review
• DxO FilmPack review
• DxO PureRAW review