https://blog.cryptographyengineering.com/2020/11/16/ok-googl...
Proof of Capture: Apple Reference Image, but open source and using steganography (merybenavente.me)
106 pointsby merybenavente14 hours ago58 comments
nmadden an hour ago
Signing things by default has repeatedly been found to have serious unintended consequences. Do you really want your leaked/stolen photos to be undeniably linked to you?
Retr0id 9 hours ago
> we sign a perceptual hash (pHash) of the image rather than an exact pixel checksum
Perceptual hashes are non-cryptographic. There are certainly collision attacks, but what about preimages? A preimage would completely break this scheme.
This paper demonstrates second-preimage attacks against PhotoDNA and PDQ: https://eprint.iacr.org/2021/1531.pdf
theamk 8 hours ago
Yep, that breaks this scheme, making it useless. But it's far from the only thing making it useless, and the github page even explicitly lists those:
- "Small content edits slip under the threshold. [...] A localised edit covering ~15%x20% of the frame [...] passes as authentic" - this is the worst part. 15%x20% is huge, for example enough to change the face of the person or the book/text on the image.
- "Cropping is not survivable, at any amount" - given the purported reason for perceptual hashing is surviving light editing, it's pretty disappointing that one of the most common light editing operation is not supported.
Oh, and the whole "cryptographic chip" angle is absolutely bogus from the security perspective. OK, attacker can't extract the private key from chip. But they can simply connect the chip to a different device and have it sign anything! Given that the attacker in this model is device owner, this is absolutely trivial.
amelius 20 minutes ago
One of the few advantages of "not owning your device".
Glad that at least we have that now.
Wendell58 35 minutes ago
A pHash isn't built for this. The 15%x20% threshold that passes as authentic is enough to swap a face.
khalic an hour ago
I very much hope that apple is using lidar data to determine if it's a flat surface being screened
fitzn 13 hours ago
Here's a toy for embedding small text into images steganographically:
https://github.com/fitzn/atrium
merybenavente 12 hours ago
cool work!
smalltorch 12 hours ago
Here's a toy for embedding text inside random HN comments.
https://gitlab.com/here_forawhile/edasm
Example:
After wholly implemented our logging layer with dagger, I posit that the real regression was not the aws itself but the rigorously prototyped around authentication. We consequently extraordinarily profiled the config, henceforth simplified every edge case, and the optimization were unmistakably exemplary. alternatively, the aforementioned monitoring is comparable advantageous to an incremental security environment. I endorse this path if your security team has rigorously instrumented a massive rust codebase before.
treyd 12 hours ago
I don't understand why few people are pointing out the obvious vulnerability here that you can control the wires going into the photosensor controller and pretend that the photosensor is capturing whatever image you want. I imagine it's not exactly trivial to do this, but a grad student with an FPGA could probably figure it out.
figmert 12 hours ago
Or, as the author said, you can just photograph an AI generated picture, and that will work too.
koinedad 11 hours ago
Adding depth sensor info to the this could help
petu 10 hours ago
This feature is Pro phones only, not Duo:
https://www.apple.com/iphone/compare/ ("Apple Reference Image (Fusion Main)")
So only on devices with LiDAR / that can capture depth map.
theamk 8 hours ago
If there is signed metadata too, then it's pretty hard. You will need to match focus distance (it will be very small if photographing picture), GPS location, exposure and other settings. If there is a depth map, you'll need to match it too.
TedDoesntTalk 12 hours ago
Even easier is to just take a picture of an AI-generated picture.
Retr0id 10 hours ago
Simpler than that, you can just talk to the cryptography IC yourself and ask it to sign stuff. No need for an FPGA, just an arduino. Given the datasheet I imagine any LLM from the last year should be able to oneshot it.
hex4def6 9 hours ago
if
I imagine on apple silicon this is buried deep in silicon / ISP IP block, and isn't a discrete IC.
Retr0id 9 hours ago
But it is, the discrete IC is pictured in the article.
whywhywhywhy 10 hours ago
>a screen attack still works: photograph a screen displaying an AI image and you get a signed photo of a fake
you don't need to do that just photograph a screen.
This seems close to worthless in "identifying real photos vs AI" for someone actually wanting to do something bad with an AI image, although probably very useful at identifying which phone took a photo when ("the root of trust stays inside Apple's Private Cloud Compute") seen as it's not an entirely local solution a bad actor government could use their powers to completely abuse this.
ares623 9 hours ago
if geolocation data can be captured in the same signature, that would be a good enough approximation for most relevant cases I think.
brainwad 3 hours ago
GNSS signals can be relatively easily faked because the original signals are very weak so overpowering them doesn't require much broadcast power.
ares623 3 hours ago
ah, damn.
altairprime 5 hours ago
Because doing so does not materially devalue Apple’s product. Sure, a dedicated attacker could try to overcome it, but few will, and only people of such serious consequence that they can afford the effort of modification. By and large this puts Apple into direct competition with Nikon and it’s long overdue that someone ship this capability to a wider market than authorities.
Also, remember how Touch ID sensors are cryptographically paired, and consider whether Apple could bake that into a camera sensor rather than a fingerprint sensor. If they can, then you can run wires all you want; the attestation chain will not be valid. I’d be shocked if they were willing to launch the product without that, and there’s a new hardware dependency or else they’d have released it for earlier phones.
vzaliva 12 hours ago
I expect in the near future all digital cameras to digitally sign the images they take. Even before AI slop, it was useful to avoid manual alterations. AI makes it all too easy, so it makes sense.
However, this will certify only the original image. I think the missing part of this is additional layers of certification which allow some image editing (e.g., rotating, contrast, etc.) yet clearly document that the image was modified and link to the original image ID. Kind of like a signed git log.
xg15 12 hours ago
EXIF data is stripped for a good reason - because it can be a privacy hazard. Suddenly this plays no role anymore?
doc_ick 12 hours ago
Ontop of this, including a photo edit history in a photo including the original photo would increase the size of a photo to be completely unusable or unshareable.
xg15 12 hours ago
I understood the GP so that only some unique ID or hash of the original image would be included, not the image itself. Basically like the commit chain of Git but without the actual content blobs.
You could use this data to prove that image B is an edit of image A if you already have both A and B.
I still think this is a bad idea, because this all requires the images to have some sort of ID - and that seems like a prime target for tracking.
lokar 11 hours ago
You can publish a fully stripped image (as people do now for exif), and retain the original
stvltvs 12 hours ago
Signing the raw image data wouldn't require also signing the EXIF metadata. For privacy, you could later strip out everything except the raw image and the camera's signature and still prove that the image is unaltered.
smalltorch 12 hours ago
I think he's saying the existence of a camera signature is the privacy issue maybe.
lokar 11 hours ago
You can remove it all.
If authenticity later becomes an issue you can produce the original.
cortesoft 11 hours ago
What are you signing it with, though? A unique key that only exists on your device... so it will perfectly tie a photo to a specific camera.
Even if you didn't know who owned the camera, you could identify other pictures taken by that same camera, and information in those photos might let you figure out who owns the camera.
hamdingers 12 hours ago
In the contexts where you need to prove an image is a real unaltered photograph (court cases, news media, science, etc) privacy is usually not a concern.
Nobody cares if your social media photos are edited, they probably are, it's fine.
xg15 9 hours ago
I'm pretty sure if signing images becomes the norm, because cameras will do it automatically then social media sites etc will start to care - or at least be thankful for the additional metadata.
theamk 8 hours ago
EXIF already includes extensive metadata, including phone model and GPS location. For professional cameras, there is often camera and lens serial numbers too.
This has been the case for many years, and so far social media sites never care - they strip that info instead. Why would this change with one more extra piece of metadata?
xoa 10 hours ago
>EXIF data is stripped for a good reason - because it can be a privacy hazard. Suddenly this plays no role anymore?
Can you clarify what your actual complaint is here? Putting aside for a second obvious implementation options, it's pretty standard that there can be a tradeoff between privacy and trust. Any of us are still perfectly free to upload images for fun to forums or social media or whatever that are private, or edited or whatever we like. But if there's something we [i]want[/i] to prove, this gives an extra option to do so. And in particular the set of cases where one cares the most about enhancing authenticity appears at first thought to be pretty much a union set with the cases where one will put their name (or at least the location and time of the picture in question) behind the image? Like, can you give examples of specific cases you are imagining where simultaneously the photo itself reveals no information about time and location [i]and[/i] it's something local/national/international-newsworthy where people would fear AI-alteration? Like, say you're photographing at a protest to document it including any violations of law. By definition, the images you take reveal the location and the time. That's the whole point of them. Having the location and timestamp signed wouldn't reveal anything extra as long as the photo was unaltered.
And actual legit journalism always has name(s) standing behind the reporting. Or for that matter, even if we're merely talking something like a review of a product, is it actually wrong to put a name or pseudonym behind that review if you expect readers to give you much credence? I mean, you'd be free in terms of law and tech to not bother. But even long before the current growing AI-slop age a lot of us have been starting to treat anonymous reviews with a lot of skepticism, or discounting them entirely for some product classes, for good reason.
Even in terms of maintaining privacy, remember we already have "tools" for that which this in turn could further help. You could privately report a tip to a reporter at a media organization, and then they could report on that without revealing you but be able to say "we verified the signature of the raw image and sensor data" alongside normal follow up. Then it's their name, but having a stronger chain behind it could still be helpful in places.
Finally getting back to "implementation options", there's no technical reason the image and multiple sets of metadata can't all be signed separately by the sensor stack such that you can pick & choose what to include and still have it all be signed, with readers giving greater or lesser weight to the trust based on your choices.
xg15 9 hours ago
I think what got me worried is the GPs take that "I expect in the near future all digital cameras to digitally sign the images they take". Sure, if you have your special, verification-enhanced camera, go for it. But if this goes like with smartphones and we have all cameras embedding such a signature - and therefore all cameras are made locked-down and tamper-resistent - then we have another category of devices removed from control of their users.
> Any of us are still perfectly free to upload images for fun to forums or social media or whatever that are private, or edited or whatever we like.
First, the paper proposed a method of embedding the metadata inside the actual image pixels in a way that makes it difficult to remove. I think it's still possible, but you're will need a specialized tool to do so and it will alter the image.
Second, that's assuming forums or social media would still be allowing uploading unsigned images. If AI images really become as much of a problem as anticipated here, then sites might decide to block any image they can't verify completely.
> Like, can you give examples of specific cases you are imagining where simultaneously the photo itself reveals no information about time and location [i]and[/i] it's something local/national/international-newsworthy where people would fear AI-alteration?
All kinds of photos from warzones have this property. We wouldn't have an entire science of geolocating photos from landmarks that were accidentally captured if everyone was so easy with putting their GPS coordinates in the file.
ranger_danger 12 hours ago
There will always be a way around it. There are even open-source alternative/hacked firmwares for professional DSLR cameras where tampering with the signing may be possible.
And this still doesn't help any other kind of image e.g. screenshots, photo of a screen etc. that can make the camera signatures largely pointless depending on the context.
stvltvs 12 hours ago
The idea is to be able to prove that a photo you took was captured by a camera, not generated. That would be helpful in the context of a disputed news story, a court case, etc.
Would that ever be relevant for a screenshot?
ranger_danger 11 hours ago
Yes in multiple ways, and there's probably more than this:
It's relevant that a screenshot doesn't have a signature, in the case that you want to remove any "proof" or tracking info from a real photo when uploading an image. Maybe I don't want people to know what brand/model of camera I use.
And it's relevant if a screenshot did have a signature if you want to "prove" that the screenshot itself hasn't been tampered with after the fact.
xg15 9 hours ago
In the method proposed in the paper, the signature is stored in the pixels and ostensibly even survives some compression. So if you made a screenshot of a signed image, the image would still have its signature inside the screenshot, but the screenshot wouldn't have any (new) signature as a whole.
That's how I understood it at least.
ranger_danger 8 hours ago
I would assume you could make big enough differences in color/contrast/brightness and especially 3D rotation (similar to taking a photo of a screen) would make such a signature unreadable... but I would love to be proven wrong.
qingcharles 9 hours ago
It's not massively helpful for a court case, IMO. This has been handled the same way since the invention of photography: ask the photographer to swear under oath they took the photo. (technically, laying a foundation for the evidence)
teravor 12 hours ago
if such a method gains traction then so will the effort to bypass it. either by stealing private keys or just projecting light onto the sensor.
mandolingual 11 hours ago
Even if it's flawed, adding any friction at all to the effort of deception is a positive step.
zvr 11 hours ago
People may be interested in the work of the Coalition for Content Provenance and Authenticity (C2PA) https://c2pa.org/
throwaway356565 11 hours ago
https://www.da.vidbuchanan.co.uk/blog/android-c2pa.html
is a post about Google Pixel C2PA cameras experiencing contact with reality
Retr0id 10 hours ago
Hi, I'm the author of the above. I also got a Sony A7 IV camera a few days ago. I already have a root shell on it. Although that wasn't even required to break Sony's C2PA implementation...
I'll write more about this in the future.
nulltrace 11 hours ago
Soon my phone can cryptographically prove the beauty filter lied at capture time.
Lammy 11 hours ago
In the future, people willingly surveil themselves 24/7 with cryptographic proof, because fake images and video will be so good that it will be the only way to prove what one didn't do. Total Information Awareness achieved :D
xg15 12 hours ago
> For example, a screen attack still works: photograph a screen displaying an AI image and you get a signed photo of a fake. But it's always nice seeing big actors interested in addressing this problem.
Yeah, very nice. So this whole idea basically doesn't work - but we get a new stealth way to embed metadata in an image that can be used for tracking...
(And a new narrative why cameras need to have TPMs and locked-down firmware as well)
ChocolateGod 12 hours ago
Couldn't the camera encode information from the depth sensor and prevent this.
TedDoesntTalk 12 hours ago
I will make miniature dioramas and photograph them.
mandolingual 12 hours ago
At that point you've earned the fruits of your deception, just like the tricksters who spent time doing physical photo editing.
petu 11 hours ago
Miniature dioramas wouldn't be size appropriate. Apple could detect faces/cars/other common objects of ~known size and verify -- or even just dump depth map for anyone to check.
petu 11 hours ago
It seems to be what Apple is doing, this feature is only available on the 18 Pro's (which have depth sensor on the back), but not Duo.
shagie 12 hours ago
Photos of photos has always been a problem.
In days of old, a Polaroid photo was considered "proof of capture".
I've got a Polaroid daylab 35 plus sitting in storage somewhere (https://www.instantoptions.com/wp/faqs/daylab/). You can project a slide through it onto Polaroid film, expose it, and have the image there.
I was also able to find a company that did slide printing. It was possible to send them a digital image and they'd send you back a slide with that image... which I then used to make a Polaroid of that image.
I had a classic 600 Polaroid photo of a UFO landing.
azatom 12 hours ago
That signature contains time and optionally gps coords, and taking photo of a screen is not simple. So overall does solve some problem
ps:most important: cam/lens settings also in the digital sig, what for a screen is different