Detecting image transparency with Swift

Zach Waugh • November 13, 2024

This is the first of hopefully many posts about some behind the scenes things in developing Artifacts, which is a new native image + link organizer for iOS and macOS, currently in beta.

When you view an image in Artifacts, we round the corners, give it a border, and add a drop shadow which looks really nice (I can say that since Jordan designed it), especially on iOS.

Almost immediately though we ran into a problem. That treatment doesn’t work well on all images, especially macOS screenshots that have a transparent background and their own drop shadow, of which I take a lot. Here’s a screenshot I had saved of some UI in the ChatGPT app for macOS. We add border over empty space and end up with double shadows.

So we need to detect which images have a built-in drop shadow and which don’t. I considered a few approaches here such as looking at the pixels in the corners and seeing if they are fully transparent, but I realized we don’t need to try to determine if the image has a drop shadow, we can make a simpler rule: does the image contain any transparency? If opaque, we apply our effects, if it contains any transparency, we disable it.

Detecting Transparency

There’s a few ways you can go about this. You can check if the image format has an alpha channel, but that’s not reliable. If it has no alpha channel, you can know it’s opaque, but the presence of an alpha channel doesn’t indicate the image has transparency. Many images will contain an alpha channel in which the alpha component for every pixel is 1. The simplest method I came up with (and if there is be a better way let me know!), was to determine the average color of an image, and if that single color has an alpha < 1, consider it non-opaque and skip our effects.

Luckily, I had come across and saved (in Artifacts, naturally) a post Christian Selig had written about getting the average color - https://christianselig.com/2021/04/efficient-average-color/. Christian’s code didn’t care about the alpha value, so I had to make just a few simple changes. Here’s the final version which is a bit simplified and condensed, but all credit goes to Christian for this. See his original blog post that has more details and his code for the comments I stripped out:

// PlatformImage and PlatformColor here are just conditional
// typealiases depending on the platform with some convenience
// methods to provide a unified api
public extension PlatformImage {
    // Via https://christianselig.com/2021/04/efficient-average-color/
    var averageColor: PlatformColor? {
        #if os(macOS)
        guard let cgImage = self.cgImage(forProposedRect: nil, context: nil, hints: nil) else { return nil }
        #else
        guard let cgImage else { return nil }
        #endif

        let size = CGSize(width: 40, height: 40)
        let width = Int(size.width)
        let height = Int(size.height)
        let totalPixels = width * height
        let colorSpace = CGColorSpaceCreateDeviceRGB()
        let bitmapInfo: UInt32 = CGBitmapInfo.byteOrder32Little.rawValue | CGImageAlphaInfo.premultipliedFirst.rawValue

        guard let context = CGContext(data: nil, width: width, height: height, bitsPerComponent: 8, bytesPerRow: width * 4, space: colorSpace, bitmapInfo: bitmapInfo) else { return nil }

        context.draw(cgImage, in: CGRect(origin: .zero, size: size))

        guard let pixelBuffer = context.data else { return nil }

        let pointer = pixelBuffer.bindMemory(to: UInt32.self, capacity: width * height)

        var totalRed = 0
        var totalBlue = 0
        var totalGreen = 0
        var totalAlpha = 0

        for x in 0..<width {
            for y in 0..<height {
                let pixel = pointer[(y * width) + x]

                let r = UInt8((pixel >> 16) & 255)
                let g = UInt8((pixel >> 8) & 255)
                let b = UInt8((pixel >> 0) & 255)
                let a = UInt8((pixel >> 24) & 255)

                totalRed += Int(r)
                totalBlue += Int(b)
                totalGreen += Int(g)
                totalAlpha += Int(a)
            }
        }

        let averageRed = CGFloat(totalRed) / CGFloat(totalPixels)
        let averageGreen = CGFloat(totalGreen) / CGFloat(totalPixels)
        let averageBlue = CGFloat(totalBlue) / CGFloat(totalPixels)
        let averageAlpha = CGFloat(totalAlpha) / CGFloat(totalPixels)

        return PlatformColor(
            red: averageRed / 255.0,
            green: averageGreen / 255.0,
            blue: averageBlue / 255.0,
            alpha: averageAlpha / 255.0
        )
    }
}

let image = //…some image
if image.averageColor.alpha < 1 {
    // Image is non-opaque
}

And the result is a nice clean presentation of all images:

You’re probably saying, but wait, what happens if the odd image here and there has a just a few pixels that are semi-transparent and really should be treated as opaque. If that happens, it’s easy enough to make a threshold for considering an image non-opaque instead of 1. In the screenshot above of the ChatGPT app, the average color alpha value is 0.7 so we could change the logic to something like alpha < 0.9 or something and would probably be a safe cut-off.