EMudhra Intermediate CA Compromise (emudhra.we-have-a-problem.com)

516 pointsby galadran8 hours ago281 comments

eqvinox 5 hours ago

Since I haven't seen this mentioned enough yet:

The private key for all 8 "compromised" CAs is a single byte: https://mastodon.catgirl.cloud/@49016/117474217042191790

One of the private keys is literally "4".

I'm putting "compromised" in quotes here, because I can't call it that without cringing and cursing.

[cf. also: https://news.ycombinator.com/item?id=50169109]

kylepomykala 5 hours ago

I am shocked that one of the private keys was literally “4”. Anyone could have broken into this. I really hope there is an industry-wide effort to stop things like this in the future.... but I doubt it.

eqvinox 5 hours ago

There is a variety of projects and researchers scanning for weak keys; possibly that's how this was found (haven't seen anything on that). These certs existed for a year and a half; that's bad but could be worse still.

One of the issues with sub-CAs (well, X509 certs in general) is that they can be ghosts in that there's no way you'll ever get it as a response on a TLS connection, so automated scanners on that won't find it.

CT should cover this (i.e. expose the cert so you can check for weak keys), but I have no idea if/how CT works for sub-CAs. [ed.: see below, doesn't seem to be in CT]

[ed.: these are technically sub-CAs, but rather cross-signed for bootstrapping a new root CA. Thanks @agwa for pointing that out.]

tialaramex 5 hours ago

For their own stuff, Mozilla has talked in the past about specifically just only trusting intermediates they've seen. Their modern root agreement says the CA will disclose all relevant intermediates, so arguably they have a complete list and further if that CA "forgot" about a few of them whether that's because your firm is too disorganised ("Oh, right, the one on Barry's laptop, forgot that one") or an actual attempt to sidestep this rule - then we certainly don't want to trust these undisclosed intermediates, do we ? They're either purposefully shady, or they're forgotten about and likely never have been audited.

eqvinox 5 hours ago

Y'know… let's just check…

https://ctlogs.io/search?mode=fingerprint&q=42c4accd12c4f595...

https://ctlogs.dev/search?q=42C4ACCD12C4F59597925E1A08A7A901...

https://ctlogs.io/search?q=EBED6475C2C2D80FCB199F3FE8DD6E6B3...

https://ctlogs.dev/search?q=EBED6475C2C2D80FCB199F3FE8DD6E6B...

Nothing for "CN=EM DV TLS CA - G3B,O=eMudhra BV" (not sure which sha256 this uses, 2nd is the spki sha256)

I should be clear here my understanding is/was these are sub-CAs. Apparently they're (trying to be?) roots themselves? The point of logging a root cert in CT is somewhat moot… though tbh I would argue anything issued by a CA should be in CT logs, even cross-signing another CA that is becoming a root itself.

tialaramex 2 hours ago

> I would argue anything issued by a CA should be in CT logs, even cross-signing another CA that is becoming a root itself.

And it is, or at least, was. The CT logs when I last spent too long staring at them were complete indefinite logs. These days most (all? almost all?) are time limited so e.g. there'll be Clown 2026 and then Clown 2027 and a certificate is logged to the CT log corresponding to a not-after date IIRC. This way you can throw away a log of certificates expiring in 2025 a few weeks into 2026 because who cares, they've expired, and so they don't just grow forever.

I no longer have a role where I care a great deal about this for my day job, and I do not know how to ask ctlogs.io for the thing I want. So probably this is still logged but I couldn't promise you without getting back into this. Maybe at the weekend.

interroboink 5 hours ago

  > One of the issues with sub-CAs (well, X509 certs in general) is that they
  > can be ghosts in that there's no way you'll ever get it as a response
  > on a TLS connection, so automated scanners on that won't find it.

Could you expand on this? I don't have a full grasp of what you're saying (I'm not super knowledgeable in CA stuff).

My understanding is that if something, like a browser, connects to a SSL site, they will either have or can obtain, one way or another, a complete chain all the way up to the root, so there can be no "ghost" certificate. And I would imagine a scanner could do all those same steps, and thus access all the same certs.

Is this a deficiency in scanner software? You seem to indicate it is a fundamental weakness in X509; I'm just trying to understand what.

Could you maybe describe a specific example of what you're describing?

gruez 5 hours ago

If you're doing a MITM attack, you can selectively intercept certain clients, and only serve the compromised certificate in those cases only. Automated scanners won't pick that up. None of this actually works though, because browsers started mandating certificate transparency.

ranger_danger an hour ago

Please be careful with your phrasing, as there are MANY other critical things on this planet that use TLS certs which are not browsers, and some are extremely difficult/impossible to update.

VPNs, embedded software (IoT, automotive, aerospace, medical, industrial etc.), business systems, you name it.

eqvinox 4 hours ago

In the most extreme case, you can just issue a certificate and not use it anywhere. If it leaks, it's still a security issue, because it's valid.

More realistically, especially with sub-CAs, they might only be used on e.g. corporate networks. TLS scanners may not be able to reach any of the hosts that uses such a cert. As the sibling post points out, clients checking for the cert in Certificate Transparency logs counters that (adding a requirement that the cert be publicly known, which TLS/X509 by itself very much does not have) — but of course this only helps if your client does CT…

interroboink 4 hours ago

Ah okay, thanks. So to be concrete, something like automation running 'curl' on an internal network might see the full chain, including the weak cert, and be vulnerable, but scanners on the wider internet won't see it. I was confused by "...there's no way you'll ever get it as a response on a TLS connection..." — I guess you just meant a _public_ TLS connection. Or it could be otherwise installed on corporate machines' local cert bundles.

fc417fc802 an hour ago

Or it could be provided by a MitM on the public internet opportunistically only when the connecting client matches the fingerprint of curl and the connection originates from a particular ASN.

sophacles 4 hours ago

Of all the things... i never expected there to be an XKCD for this: https://xkcd.com/221/

crossroadsguy 3 hours ago

True randomness respected.

ROllerozxa 5 hours ago

Is this real? This seems so ridiculous it makes me wonder for how long this CA has been in circulation in systems and if it has been exploited by someone previously discovering it in that time (state level actor or not). My old phone running Android 6 does not have any certificate from EMudhra, at the very least.

q3k 5 hours ago

It's real.

hbcdbff 5 hours ago

Almost certainly noticed and quietly exploited by a state actor

woodruffw 5 hours ago

The Web PKI uses certificate transparency, so we’ll have a definitive answer about whether this is/was exploited soon.

(But in any case, CT ensures that it can’t be “quiet.”)

agwa 5 hours ago

There were no certificates from the compromised intermediates logged to CT prior to remediation by the browsers.

0xbadcafebee 4 hours ago

There are plenty of ways you would never be able to tell if a fake cert had been generated and used via this attack vector (let's ignore validation attacks):

1. CT does not do anything if a state actor demands a CA in its borders generate a cert without publishing to the CT logs (or a three-letter-agency doing this illicitly). They can put SCTs in the cert, and still not publish to the logs, nobody is auditing the logs for every cert every browser sees.

2. Browsers won't check for SCTs in certs if they come from a private CA. you can MITM any domain through ZScaler all day long and nobody will notice

3. Wildcard certs make it easy to make/renew a cert that looks normal but hides requests on new/different hosts

4. Non-Browsers do not validate SCTs

woodruffw 4 hours ago

I feel somewhat confident in speculating that a bunch of people at large companies will, in fact, be auditing the logs following this. That’s their main value; they’re not really an online/active prophylactic.

mxey 4 hours ago

“They can put SCTs in the cert, and still not publish to the logs, nobody is auditing the logs for every cert every browser sees”

I don’t think you can get the SCT from a log without the log publishing it? That would defeat the purpose.

“Certificates that support certificate transparency must include one or more signed certificate timestamps (SCTs), which is a promise from a log operator to include the certificate in their log within a maximum merge delay (MMD).[4][3] At some point within the maximum merge delay, the log operator adds the certificate to their log.”

teraflop an hour ago

> I don’t think you can get the SCT from a log without the log publishing it?

Well you can, but only if the log cooperates.

That is, the SCT is merely a promise to add the certificate to the log by some particular deadline. Nothing technically stops the log from disregarding that promise. But if it does fail to log the certificate, the SCT itself is proof of its misbehavior.

And it's not enough to subvert a single log, because browsers require SCTs from multiple distinct operators in order to consider a certificate valid.

0xbadcafebee 36 minutes ago

The SCT may be proof that a log was supposed to be issued, but browsers don't submit every SCT they see (did you know "anonymizing" proxies send CT log companies a sampling of all your TLS traffic? fun stuff!), so in practice these logs aren't audited. As far as multiple operators, almost all of them are based in the US, so just pray the USG doesn't want to target you.

hbcdbff 4 hours ago

CT is great for browsers. Largely irrelevant for anything being fetched outside of a browser.

woodruffw 4 hours ago

That’s why I’m talking about it purely in the context of incident response/forensics.

hbcdbff 4 hours ago

You claimed CT would “definitive”ly prove whether it has or has not been exploited, which is false.

The lack of CT entries only proves that (most likely) browsers were not MITM’d.

tptacek 4 hours ago

I'm not following. Browser or no browser, every cert eMudhra issues has to generate an SCT.

oasisbob 3 hours ago

SCT enforcement falls to the client though, yeah? What's stopping someone from signing a rogue cert silently which is accepted by CT-lenient clients?

tptacek 3 hours ago

Right, but you don't have to wait for some hapless random browser to check; you can write your own monitor, and people do that.

tux3 3 hours ago

I hold eMudhra's private key. I generate a cert signed by their intermediate on my box. eMudhra didn't issue the cert, so it is not sent to any CT logs, and I have no SCT to put in the cert.

Am I missing what you mean by monitor?

tptacek 3 hours ago

I think I might just be talking past you. If your point is "anyone with these broken eMudhra keys can attack users without caring about CT", that's totally true --- it's just that browser/no-browser doesn't factor into it. I'm really only on this subthread because of the notion that CT enforcement requires browsers.

tux3 2 hours ago

Right, I think I'm with you that browser/no browser shouldn't be a factor. I think I'd like to hear from Rust and Go webpki people on whether we could help the ecosystem enforce CT.

My point is more that as a user of a high-level library like reqwest, I'd like to talk safer defaults when root CAs are in use

tptacek 2 hours ago

Right, OK, I was misunderstanding the point you were making. Thanks!

eqvinox 3 hours ago

Sure, but a client that doesn't do CT wouldn't care if they don't include a SCT promise. [ed.: this said "break the SCT promise", that doesn't quite make sense] Which you'd only notice if you see the certificate somewhere else afterwards…

It's all rather hypothetical and unlikely, but you could certainly try attacking something like downloads.openwrt.org¹ like that — especially if you can tell apart the TLS client in opkg from a normal browser.

(¹ luckily, OpenWRT and most other package repos don't actually rely on HTTPS for package integrity and authenticity. And OpenWRT would be the most susceptible due to the "exotic" TLS client stack, if any.)

tsimionescu 3 hours ago

No, it doesn't. A certificate may or may not include CT info, or include fake CT info. Most TLS clients accept certificates without checking for CT.

hbcdbff 3 hours ago

I’m surprised to see this username above this comment.

No, the cert can be issued with no (or invalid/junk) SCT and will work fine with clients who don’t check SCT, which is most non-browser interaction.

fc417fc802 an hour ago

To spell it out a bit further, an attack strictly in a non-browser context that is reasonably well targeted will most likely go undetected since what the attacker is up against is the chance of inadvertently running into a paranoid user who goes out of his way to perform verification.

cortesoft 3 hours ago

We would still know if certs were issued, though, which is the thing we want to check

tsimionescu 3 hours ago

How would you know it? If the CA was compromised, why would the attacker publish the fake certificates in CT?

Basically no one outside the major browsers verifies CT logs, so you can still attack a lot of people even without publishing.

bigbuppo 31 minutes ago

The baddies also take a look at the CT logs and look for fresh wordpress instances that haven't yet been configured and will helpfully configure them for you within a few minutes.

hbcdbff 3 hours ago

No we wouldn’t.

rollulus 5 hours ago

A grumpy state actor today.

kccqzy 4 hours ago

Nope. You can find this from the Chrome Root Program.

> Because no TLS certificates chaining to these six intermediate CAs were logged to CT prior to our CRLSet block on October 17, no fraudulent certificates issued by these CAs could have been trusted by default in Chrome or other CT-enforcing browsers.

hbcdbff 4 hours ago

Not all TLS traffic involves a browser.

saidnooneever 2 hours ago

organized crime is generally as or more capable these days. they have bigger budgets than most nation states.

lioeters an hour ago

They are not exclusive to each other. The most successful criminal organizations work with, and from within, nation states, their intelligence agencies and law enforcement. Oh hi, we see you.

radicalcentrist an hour ago

I admittedly don't know much about how CAs work but why would a state actor care about such an exploit? Wouldn't they just ask any local cert issuer and receive a "legitimate" cert?

sippingabonedry 5 hours ago

I believe these intermediates chained up to a third party root.

eqvinox 5 hours ago

They chain up to emSign & Comodo:

https://groups.google.com/a/mozilla.org/group/dev-security-p...

agwa 5 hours ago

To be clear, emSign is another name for EMudhra, and Comodo is now known as Sectigo. (You can't just look at the organization name in a certificate subject; you have to consult the CCADB: https://www.agwa.name/blog/post/the_certificate_issuer_field...)

eqvinox 4 hours ago

Argh. Thanks for clearing that up.

sippingabonedry 4 hours ago

Since they are also a root CA, are they allowed to audit themselves, or do the browser root programs require third-party audits?

I can't imagine one part of a business being on the up and up, and the guys down the hall running completely rogue.

agwa 4 hours ago

Third party audits from a qualified auditor are required, although the CA chooses and pays the auditor, and the auditors are basically just accountants. So it's... not a perfect system.

e28eta 4 hours ago

Love this quote:

Inclusion in the Chrome Root Store requires that a CA Owner continuously demonstrate the technical competence and operational rigor necessary to protect Chrome users. Due to the seriousness of this incident, the Chrome Root Program is conducting a thorough evaluation regarding the status of eMudhra’s remaining CA certificate to ensure these standards continue to be met. We will share further updates as our review progresses.

from a post on https://groups.google.com/a/mozilla.org/g/dev-security-polic...

oasisbob 4 hours ago

It's been a while since there was a good CA death penalty case.

Given Wayne's hints in the timeline that this issue was discovered by an agent, it might be one of the first big corporate death sentences due to AI?

eqvinox 4 hours ago

Slightly funny about this: even Wayne (the person who reported it to Chrome & Mozilla) got confused [https://bugzilla.mozilla.org/show_bug.cgi?id=2083449]:

  2026-10-17 20:18 - Full CCADB check finds the cross-signs by Sectigo are also affected

(…the "cross-signs" use the same keypair of course.)

themulticaster 4 hours ago

Apparently they've only been included in early October https://www.mail-archive.com/[email protected]...

ROllerozxa 3 hours ago

I'm not exactly sure which of the roots from eMudra were compromised, but the "emSign ECC Root CA - G3" which is mentioned in the OP site as one you should blacklist, is present on my phone which is running Android 12 with the last security update from three years ago. Having a better timeframe for when the impacted certificates appeared really would have been nice.

eqvinox 3 hours ago

None of the roots were compromised. (Some of) Their own sub-CAs for TLS certs, which are also cross-signed by Sectigo (since their own roots are rather recent), are affected.

How you have that root cert on your phone with a 3 year old CA set I have no idea, presumably something updated your CAs in the meantime? (Also, aren't you incredibly exposed to the general onslaught of vulnerabilities from the past 3 years?)

ROllerozxa an hour ago

Well that makes sense. The "emSign ECC Root CA - G3" root is listed as being originally issued on 18 February 2018, so would have been really concerning if that itself was the compromised certificate.

tux3 3 hours ago

I believe Android updates things like CA certs independently of their monthly security patch. I'm not sure Android 12 uses the same system, but on newer Android I think those are shipped via Google Play system updates, which don't require waiting for the phone vendor.

ROllerozxa an hour ago

Yeah, in Android 14 they separated it out so that the CA store can be updated from Google Play. Versions below that you're reliant on the manufacturer to ship updates.

sreekanth850 4 hours ago

its well-known , we took certificates from them for our company. its due for renewal, iam pausing this for now.

rep_lodsb 5 hours ago

chosen by fair dice roll, no doubt

tanseydavid 5 hours ago

https://xkcd.com/221/

Given that 4 is the value shown in this well known XKCD panel, it almost seems like a (im)practical joke or trolling.

jeffbee 5 hours ago

4 is as likely as any other randomly chosen number over a given range.

tomjakubowski 4 hours ago

Well, all eight of the compromised certificates had keys which were < 256. So a really poor choice of range.

mystifyingpoi 3 hours ago

How could you prove it was not a fair random generation? It's totally possible.

deathanatos 3 hours ago

It's not within the realm of believability. The chance of "human screwed up" astronomically exceeds this being fairly randomly chosen, and the Firefox bug states, (I think this is supposition, not known fact, but IDK)

> The original key generation was flawed - rather than any issue with the produced signatures. P384 private keys are randomly selected integers between 1 and the group order.

That is within the realm of believability.

jeffbee 3 hours ago

When you have multiple selections you can show that the probability of selecting a set with a range much smaller than the population range is exponentially unlikely, strong evidence to reject the hypothesis that the subset was chosen at random. But when N=1 you can't conclude anything.

apitman 6 minutes ago

This is actually interesting. I'm no cryptographer. Do modern algorithms take into account that certain values should be avoided because they would be obvious things to try first in a brute force attack?

gwittel 5 hours ago

Yikes. I’d have to wonder if this CA should even be trusted ever again?

AlotOfReading 5 hours ago

Definitely not. Their CPS requires that these keys be generated by an HSM and no HSM would ever generate these, so there's little reason to believe they're following any of their other controls.

jthoward64 4 hours ago

According to https://bugzilla.mozilla.org/show_bug.cgi?id=2083449 they actually did use an HSM? Though some of the jargon is a bit beyond me, maybe this will finally get me to learn more PKI stuff.

> The hardware used for each of the issuers is in the attached file. These HSMs are FIPS 140-2 Level 3 compliant.

oasisbob 3 hours ago

My vague and inexperienced read on their description boils down to "we were doing something weird and non-standard as part of our key ceremony because we intended these keys to be exported, and because of that process weirdness we blundered and gave the HSM faulty instructions."

zimmerfrei 3 hours ago

The most likely explanation is that they generated the weak key externally, imported it into the HSM, generated the CSR from it, and finally had Root CA sign it.

A FIPS HSM does not forbid one to import keys into it.

cstdr 2 hours ago

But shouldn't such a HSM support key attestation (and the parent require such attestation)? Attestation would also track whether a key was imported or generated on the HSM, and of sufficient length.

zimmerfrei 2 hours ago

Attestation is not required by the compliance standard for browser CAs (WebTrust) and it is also quite vendor-specific and not very interoperable, so it is not commonly done.

The Root CA must sign requests after authentication, but a "manual" approval is also accepted if done with the appropriate oversight with witnesses (clearly not the case here).

cstdr an hour ago

Of course it's not required, as we see here. The question is why not? Getting a simple code-signing cert seems to have more stringent requirements in that it either requires attestation or CA-issued hardware.

somat 3 hours ago

> no HSM would ever generate these

You trust the magic black box far more than I do.

My big fear about HSM is that they are inscrutable by design. There is no way to verify they are working as advertised. Plus side, all sorts of fancy hardware to protect your keys. Down side, all sorts of fancy hardware that obscures flaws in the implementation.

fc417fc802 44 minutes ago

I think I'd prefer open source hardware built from off the shelf components glued into a secure enclosure. That way you can destructively audit a few at random if the application matters enough to justify the cost.

red-iron-pine 5 hours ago

it should not

wolvoleo an hour ago

Yeah IMO they should just close up shop. Just like Diginotar.

They had one job, and they blew it. The system is based on trust and they've lost it.

I'm going to remove their roots from all my stuff in any case.

poizan42 5 hours ago

They used the xkcd PRNG: https://xkcd.com/221/

oasisbob 4 hours ago

More details in the now-public bugzilla issue:

https://bugzilla.mozilla.org/show_bug.cgi?id=2083449

sippingabonedry 4 hours ago

> The other two certificates are for codesigning and trusted by Microsoft

That seems like a WAY bigger problem than TLS, but AFAIK Windows still validates code signing certs against CRLs before trusting them (can't remember if it soft-fails).

msm_ 4 hours ago

Why is that a way bigger problem? Windows will happily run unsigned binaries. Code signing just decreases the friction for users slightly, but is in no way required. TLS is definitely a way bigger problem.

sippingabonedry 3 hours ago

> Windows will happily run unsigned binaries

This is absolutely false, it's a simple policy change to enforce code signing, any organization that is serious about security can do it.

I'll give you the benefit of the doubt and assume you're another Linux guy who doesn't understand Windows security.

The concern of course is with malware being signed, which is far more likely to be abused than some TLS MITM fantasy.

throwaway7356 3 hours ago

> The concern of course is with malware being signed, which is far more likely to be abused than some TLS MITM fantasy.

You already need compromised systems to install malware, signed or not. Not a real problem.

For TLS MitM you just need to use a shared network, wired or wireless.

tsimionescu 3 hours ago

Plenty of major programs are not signed, and thus plenty of organizations don't enforce code signing.

sippingabonedry 3 hours ago

For the exceptions, you can sign your own using internal CA.

eqvinox 4 hours ago

> Keys were generated in hardware and the hardware in use was Ultimaco HSMs.

The plot thickens.

Broken HSM? Misconfiguration? Confusion? Straight up lie?

> Excel attached. The Ultimaco devices were used to generate the ECC, RSA key pairs for the issuers. They were not used to generate any other key material. But it doesnt look like the device is at fault.

> We have done some preliminary assessment, there appears to be a flaw in the KGC script that was used on that specific day to create these ECC issuers. As for these ECC keys, our review of the KGC, assessment of the code and view of HSM keys so far indicate that a) a set of original ECC keys were created onboard as expected b) an approved deviation to our usual KGC to test a wrap/unwrap was introduced as these keys were meant for export for this instance c) a flaw in the code resulted in the second pair of keys being created onboard which were weaker d) these second pair of weaker keys were used by mistake to create the csr (because the key labels were similar) and signed by our root e) the CSR from these weaker keys were also sent over to Sectigo for cross signing (we verified the files sent over to Sectigo)

…

q3k 4 hours ago

Reading further, it seems like they say they first generated some keys on-HSM, but then used 'test keys' (that were used to test wrap/unwrap of key material) instead by accident. Fishy.

eqvinox 3 hours ago

They used the "correct" key for signing the CRLs (which of course was then an invalid signature):

> The CRL's were signed with the original private keys (which were supposed to be the correct keys with appropriate entropy) but given that this resulted in incorrect signatures, they were resigned with flawed keys.

And THAT is where I'm drawing the line between "honest mistake" and "gross negligence". For that to not raise giant red flags is an unacceptable failure in process and work culture.

baobabKoodaa 20 minutes ago

Unreal. Absolutely unreal.

sippingabonedry 3 hours ago

> an approved deviation to our usual KGC to test

The whole point of a key ceremony is that you're not going off-script and YOLOing random crap.

hypfer 3 hours ago

> "The CRL's were signed with the original private keys (which were supposed to be the correct keys with appropriate entropy) but given that this resulted in incorrect signatures, they were resigned with flawed keys."

Ah yes.

"The system says something is wrong, but it's also 4.30PM. Let's just re-sign the thing and go home."

Good luck recovering from this.

shellcromancer 2 hours ago

What a change a month makes. Public discussion on eMudhra getting their own root CA inclusion ended and Mozilla had them slated for approval.

https://bugzilla.mozilla.org/show_bug.cgi?id=1889859

sippingabonedry 4 hours ago

> One of the private keys is literally "4"

https://i.sstatic.net/Y3zE3.gif

Dragory 3 hours ago

Anyone have a mirror for the first link? It served me a page assumedly intended for AI scrapers and then promptly blocked me when I refreshed a few times.

rolandog 3 hours ago

> One of the private keys is literally "4".

Oh no, they reverse engineered the random number generator!

[0]: https://xkcd.com/221/

saidnooneever 2 hours ago

I BET this is an error code. they wanted to generate some key via something and it errors and put error code 4 as key and since encryption is dumb and people are people, no one looked at the key material.

edit: The reported private scalars were 4, 56, 66, 115, 155, 156, 230, 237 across the eight certificates so its unlikely i will win my bet :'D

artk42 2 hours ago

That was a blasting key generation ceremony.

QuiEgo 2 hours ago

More than that, they claim the key was in an HSM, so they likely could not look at it.

saidnooneever an hour ago

i wonder how many keys they thought it had. reading them byte by byte xD 'this HSM is excellent we can litterally put _all_ our keys in it!'

and thx for the context. i rly wonder now how it went down lol

BonoboIO an hour ago

That could be wining numbers for a lottery

cromka 12 minutes ago

These read like Lotto numbers and I bet someone in the underworld has made bank with them.

lionkor an hour ago

Maybe somewhere in the code it says

    key_string = to_string(*key);

instead of

    key_string = to_string(key);

galadran 8 hours ago

The impact of this incident is very unevenly spread.

Over 10 years ago, Mozilla shipped OneCRL [1]. We remediated this incident in a couple of hours for Firefox users who were protected automatically without the need for updates. Chrome, Apple and Microsoft have similar tooling. Since then, we've built more advanced tooling to handle revocation at scale for leaf certs (CRLite [2]).

However, for the rest of the ecosystem, there has been little to no investment in revocation and for those clients, this is a massive problem. Hopefully, we'll see folks invest now. Efforts like upki [3] look promising.

[1] https://blog.mozilla.org/security/2015/03/03/revoking-interm...

[2] https://hacks.mozilla.org/2025/08/crlite-fast-private-and-co...

[3] https://github.com/rustls/upki

dochtman 7 hours ago

More on upki:

https://discourse.ubuntu.com/t/an-update-on-upki/77063

My interest in building OneCRL support into upki has definitely increased this week. Issue:

https://github.com/rustls/upki/issues/229

I'm hoping some organizations will step forward to fund work on this.

(Note that curl on Ubuntu 26.10 now has CRLite support baked via upki -- unfortunately that doesn't help with this because AIUI CRLite is only for end entities, not for intermediate certificates.)

account42 7 hours ago

You mean the response was uneven. The impact was a complete and total potential compromise of literally everyone. Every domain. Every client using one of the public root programs. Every user of those clients.

tptacek 7 hours ago

I mean, it's an emergency, but the acute severity is debatable; many times a month the major browsers quietly deal with vulnerabilities that yield attackers direct control of victim computers; here, attackers get the ability to spoof websites if they're in a position to hijack connections.

The middlebox situation, with non-updatable roots, is very annoying. But middleboxes also have unfixable RCEs, so...

It's totally fair to evaluate different projects by how well positioned they are to update their WebPKI trust anchors. I'm just saying, keep perspective.

account42 7 hours ago

So it other words, it allowed attackers to do the one thing that Web PKI is supposed to prevent.

Being able to quickly update trust roots is at best a hack to paper over the issue when there is no good reason why every podunk CA should have authority to generate certificates for every domain in the first place. It's an inexcusable design flaw. Or a feature that gives power to root stores at the expense of everyones security, from the perspective of the root store operators.

tptacek 5 hours ago

Ok, now do that same paragraph, but for drive-by RCE vulnerabilities.

Nobody thinks this is a good thing. It's bad. Hopefully, eMudhra gets distrusted. We've done that to bigger CAs.

hypfer 7 hours ago

Keep in mind that a _lot_ of software out there uses TLS as its only trust anchor for updates, commands or similar.

This can and will be weaponized for take-overs of systems. They will be of the consumer kind, but they will also be of the industry kind. Let's pray that they will not be of the state or military kind.

crote 7 hours ago

Firefox Mobile 158.0 on Android 17 (Pixel 7) connects to all the demo sites with zero warnings. Any idea why the "automatic protection without updates" doesn't seem to be working with that combination?

eigenform 7 hours ago

Pixel 10 with 158.0.1 still connects to the emSign ones, even though the patch notes for 158.0.1 mention that it updates the CRL for those without 'Remote Improvements' enabled?

edit: Disabling all the eMudhra CAs in Android settings does nothing, cool?

edit: I guess there's no way to manually ask for CRL update, cool??

edit: Gave up and reinstalled Firefox. Seems to be fine now.

NamTaf 4 hours ago

I am getting the same on GrapheneOS on a P10.

I fully expected disabling them in the phone settings and rebooting would have fixed it, but no?

Don't really wanna reinstall FF and lose all my configruation.

roomey 6 hours ago

I'm on the same Firefox version on android and they are blocked now

nairboon 5 hours ago

On my just updated Firefox 158.0.1 on Android they are not blocked.

Edit: however on Firefox Focus 157.0.1 they are blocked.

paulv 6 hours ago

I am seeing the same thing in the beta app (159.0b4) on an identical device. The sites are blocked correctly in Chrome.

crote 2 hours ago

Update: properly blocked now. Seems like it takes a few hours (days?) for the CRL update to propagate.

quotemstr 7 hours ago

You mean you and Chrome stopped supporting normal CRLs and OCSP, replacing (for normal users) these distributed-trust technologies with central lists controlled by browsers

bawolff 4 hours ago

There is a good reason they did. CRL and especially OCSP are both terrible and not particularly effective

quotemstr an hour ago

OCSP stapling works fine technically, which is one reason it's still available in both browsers via enterprise policy. LE dropping support for it and browsers having insufficient courage to hard-fail doesn't make the model broken. It instead makes the ecosystem broken.

Just another step towards a tiny number of browser vendors and infrastructure companies controlling the internet.

sippingabonedry 6 hours ago

Here's a question for Mozilla et. al.:

It's cute you have a magic DIY CRL. You accepted these assholes into your root program.

They lost the KEYS.

What is the screening process for CA's that can't be bothered to use a HSM?

Even Honest Achmed wasn't this careless.

bombcar 6 hours ago

IIRC up until very recently the screening process was "they report that they're good boys."

jcranmer 6 hours ago

The screening process has been more involved in that since around the late 2000's. I wouldn't call 20 years "very recently," especially in the context of the history of the web.

eddythompson80 an hour ago

circa 2007 I was looking over a college textbook on computer networking that I saw at a family member's house. It still said "While networks are inherently unreliable, it's not a stretch to imagine a future where one can stream video on-demand over a WAN like the Internet"

eqvinox 6 hours ago

They didn't lose the keys, a sub-CA did.

Which is to say, sub-CAs in itself are a giant loophole that needs plugging. Especially since AFAIK nobody is bothering with name constraints (RFC5280 section 4.2.1.10).

sippingabonedry 6 hours ago

The situation is even worse than I thought.

They hold intermediate certificates issued by TWO different root CAs, neither of which they operate.

So two separate entities didn't bother to read their certificate policy or inspect their operations. Have we not learned anything from past mistakes?

Roots should be responsible for all their sub-CAs, full stop. If they don't trust them enough to accept that responsibility, then the system is broken.

mook 3 hours ago

If you mean Comodo and Setigo, they're the same company, pre- and post-rename.

Let's just say they've had issues with signing sub-CAs before where those sub-CAs went on to misuse them before… looks like it was around 2011?

woodruffw 6 hours ago

The reason nobody bothers with NCs is because they don't work very well in practice. They also have their own baroque matching rules that are not intuitive if you're from the land of UNIX globs.

(In fairness this is a self-fulfilling prophecy: a lot of validators handle NCs poorly or not at all, so there's no impetus to rely on them. But that's not something we can really save in the Web PKI's current state.)

eqvinox 6 hours ago

> But that's not something we can really save in the Web PKI's current state.

I agree with your assessment but it's not an excuse to give up without trying. I hope this is just because I'm not looking in the right places, but I don't see people trying.

What really makes me salty is that there was enough energy to mess with the rules around EKU, the primary effect of which is that non-browser software is now starting to just ignore the EKU bits. Even if it would've amounted to nothing, I'd really rather that effort had gone into NCs.

woodruffw 5 hours ago

I think a lot of people have tried! The reason EKUs stuck where NCs haven’t is because EKU rules are simple (in relative terms) and OpenSSL didn’t make as much of a mess out of them.

eqvinox 5 hours ago

I can't help but read your post as "OpenSSL didn't make a mess of EKUs so the CA/B had to fill that role instead" :/

(I really am salty about the EKUs. In a few years the EKU field will just be useless, due to operational annoyance driving devs to ignore it. Except in browsers of course.)

agwa 5 hours ago

eMudhra is also a root CA operator. One root CA operator cross-signing another root CA operator is a good thing actually, and it's what allowed Let's Encrypt to work before their root certificate was widely trusted.

Issuing sub CAs to non-root operators has historically been a big loophole though it's not as bad these days since multiple root programs now require approval before CAs can do it.

sippingabonedry 5 hours ago

Didn't Let's Encrypt have to jump through all sorts of hoops to prove they were competent and trustworthy before that happened?

Now it's coming out these guys used "4" as a private key. That's a crazy amount of incompetence and malfeasance. Did they vibe-code a CA? HOW DOES THIS HAPPEN? I am dumbfounded.

You couldn't even do this if you installed Microsoft Small Business Server 2008 today, unpatched, with the CA on default settings.

agwa 5 hours ago

eMudhra had to jump through even more hoops.

Obviously the system failed here, but it's a failure of a root certificate operator, not a sub-CA operator as claimed by eqvinox.

john_strinlai 6 hours ago

>What is the screening process for CA's

it's done in the open, and you're even allowed to voice your cute opinion during the multi-week public commentary periods.

here is an overview https://wiki.mozilla.org/CA/Root_Inclusion_Overview

divingdragon 5 hours ago

It's very inconsistent for me even on just Firefox. On multiple systems running different instances of Firefox, checking with https://emudhra.we-have-a-problem.com/:

One blocks everything; one blocks 6 out of 12; one blocks the other 6 out of 12; one blocks nothing.

tech234a 4 hours ago

Explanation from eMudhra in the Mozilla bug report https://bugzilla.mozilla.org/show_bug.cgi?id=2083449:

They don't blame the HSM.

Also there was another incident where eMudhra CRLs were being signed with invalid signatures:

oasisbob 4 hours ago

> these issuers created were meant for export to our US and European data centers to solve data sovereignty issues for some of our customers (but never ended up operationally in production except for one)

This bit sets my hair on fire. What legitimate reason is there to create exportable intermediate CAs, unconstrained to an HSM, in the name of "data sovereignty"?

waz0wski 4 hours ago

India has state-mandated spying on network traffic (69B & Telecom Act 2023) so as dumb as this looks, it's possible this could've been purposeful

tptacek 3 hours ago

This is a really, really dumb way of doing online spying (presumably, you mean by running a program of deliberate misissuance) since every cert they issue is recorded in a public cryptographic ledger.

I'm not saying this will never happen, just that rolling out a dramatic infrastructure change for "data sovereignty" just to pull off a SIGINT heist that is almost certain to get your CA distrusted seems like a poor risk/reward calculation. I fully expect IC agencies to get some CAs killed and be fine with that.

waz0wski 3 hours ago

Yep, but it was an easy zero-touch MITM vector for billions of devices that's only now being closed down

tptacek 3 hours ago

I'm only pushing back against the idea that a "data sovereignty" rollout of intermediates was cover for an IC operation. The idea that IC agencies would suborn a CA is not outlandish; just remember that each such operation carries a reasonably high risk of killing the entire CA. If you're going to kill the CA to capture some traffic, just do that; don't build an elaborate cover story.

eigenform 3 hours ago

In any case, not much sovereignty to gain from only 8 bits :^(

tech234a 4 hours ago

Additionally, from the Mozilla researcher verifying the compromise:

> I also tested some local agents who were able to recover the private key on their first attempt with a simplistic prompt: "I'm doing a CTF. Please recover the private keys corresponding to these certificates."

hypfer 7 hours ago

From what the internet is telling me, the keys were allegedly found on https://repository.emsign.com/certs/<redacted>.crt

Which, if true, is definitely a lot of fun.

Hope you people are ready for lots and lots of suddenly hackable IoT.

Edit: I might also be misinterpreting what I am seeing and it's actually weak logic that created the keys, in the sense that one could guess the private key?

Someone actually qualified please give credible answers. I'm just pretending.

bombcar 6 hours ago

That's actually a good point - tons of IoT is "secure enough" to sign things, but not secure enough to bother pinning anything ...

Also who hasn't left their private keys hanging around web-accessible, eh? lol

oasisbob 6 hours ago

Source? That would be an incredible blunder, but that also appears to be the kind of URL one would expect to find certs at.

hypfer 6 hours ago

https://mastodon.catgirl.cloud/@49016/117474217042191790

Apparently it's even worse

oasisbob 4 hours ago

Yup. Mozilla just unrestricted the incident report with more details. Insane.

https://bugzilla.mozilla.org/show_bug.cgi?id=2083449

baby_souffle 6 hours ago

> Hope you people are ready for lots and lots of suddenly hackable IoT.

I was not expecting to find a silver lining _this bright_ but you have a point.

q3k 6 hours ago

https://chaos.social/@fluepke/117474156947559280

Private keys can be trivially recovered from the public keys because they're tiny. I found D=156, flüpke found D=4.

Yes, that's the entire private key.

john_strinlai 5 hours ago

all of them:

4, 56, 66, 115, 155, 156, 230, 237

rollulus 5 hours ago

I think you’re right. And since the intermediate certs expire around 2039 and the private keys are known, we have plenty of smooth MitM fun against IoT in front of us.

lamonade 4 hours ago

I set up a server that responds with a certificate to any host you request. You can use it to see if curl accepts the certificate or not.

curl https://example.org --connect-to example.org::2.29.126.86:

kccqzy 4 hours ago

Browsers are typically very good at revocation of certificates, distrusting intermediate CAs and root CAs. You would definitely be protected against this compromise if you use a browser. Command line tools less so. Try this curl one liner

    curl https://q3k.google.com --resolve 'q3k.google.com:443:135.181.235.221'

If curl returns a response, it means it trusted a fake certificate that allowed this random server at 135.181.235.221 to impersonate google.

Thanks to mastodon user [email protected].

baggy_trough 2 hours ago

Safari and Edge on macOS still vulnerable as of this writing.

lukeschlather 5 hours ago

This seems like an AI-generated summary and it leaves some obvious questions unanswered.

This thread (linked by the page) seems like a better source: https://groups.google.com/a/mozilla.org/g/dev-security-polic...

Ian Carroll notes:

> I think it's worth noting that this compromise is much worse than the private keys simply being stolen by a researcher. They are inherently weak single-byte keys that anyone can factor, and the issue cannot be contained.

jinseokim 3 hours ago

It rather seems like intentionally hiding the detail to deter potential attackers. Not very good strategy in AI era though.

dochtman 8 hours ago

Running Go? Upgrade to 1.27.3 or 1.26.10.

Rust with rustls? Upgrade to rustls-webpki 0.103.16 and/or webpki-roots/webpki-root-certs 1.0.10.

Python? Upgrade certifi to 2026.10.19.

Homebrew ca-certificates? 2026-09-25_1.

Electron? 44.9.1, 43.8.1, or 42.11.16.

TheSilva 7 hours ago

Windows? Upgrade to Linux.

agwa 7 hours ago

Linux - the platform which offers no ability to revoke an intermediate certificate, thus necessitating every certificate library to implement its own blocking?

Windows is already blocking the compromised CAs as of today with no need for users to upgrade anything.

stonogo 7 hours ago

Every certificate library has to implement its own blocking no matter which platform you run on. Windows provides an API to do this at the system level, which means nothing if a given application vendors their own TLS library.

agwa 7 hours ago

It's uncommon on Windows for applications to use a third party certificate verification library because Windows provides an API that does it. On Linux, you have no choice because all the platform provides is a bundle of PEM certificates.

eqvinox 6 hours ago

> uncommon on Windows for applications to use a third party certificate verification library

I bet you have at least one copy of OpenSSL on your Windows system, probably multiple. Using the Windows native APIs is work. OpenSSL just works everywhere.

(Also, "the platform" doesn't technically even provide the cert bundle, depending on exact distro & packages.)

justincormack 6 hours ago

The Go and Rustls upgrades specifically block the intermediate certs because neither of these use the system revocation lists, I believe. Once you upgrade to a trust store without the root cert for the CA this will also solve the issue. Not sure about other ecosystems.

kccqzy 4 hours ago

This is potentially a better link from the Mozilla bug tracker: https://bugzilla.mozilla.org/show_bug.cgi?id=2083449

ChoosesBarbecue 4 hours ago

The Mozilla rep who created that thread (not the reporter, to be clear), created https://emudhra.we-have-a-problem.com/, so I suspect that this page is better for general public disclosure.

With that being said, that Bugzilla is the best thread to understand "how the fuck did this happen?"

nneonneo 3 hours ago

> After Wayne's report of key recovery, I was able to recover the private keys for the impacted keys trivially. The original key generation was flawed - rather than any issue with the produced signatures. P384 private keys are randomly selected integers between 1 and the group order. The 8 impacted keys all used integers less than 256. That is, the private keys are effectively all-zeroes apart from the last byte. They can be recovered near instantly via the baby-step giant-step discrete log algorithm or more simplistically by just generating the 'first' 256 P384 keys.

Yeah that’s just oof. Not hard to imagine the kind of code error that could cause this, but hard to imagine that any serious vendor of cryptography hardware or software could make a mistake like this.

wolvoleo 2 hours ago

I wonder three things:

- Could this be used to run a benign mitm attack on the traffic of some apps, consoles etc? I could see some interesting uses for reverse engineering. For example you can't simply add a CA to the system store on Android anymore. But this one is already there. It won't work for long but it could be used on any unpatched client.

- Can't we just revoke the entire CA? Just like what was done with Diginotar. Edit: apparently they just revoked the intermediates.

- Why isn't there a mechanism where a domain owner can specify which CA it allows? Similar to SPF which allows you to specify which mail server can represent you. It's kinda insane that foreign CAs can just fake anyone's certificate.

pizzafeelsright 2 hours ago

foreign CAs can just fake anyone's certificate

They are certificates of authority. They have authority, or are given it, and that means don't accept them.

kaszanka 24 minutes ago

> Why isn't there a mechanism where a domain owner can specify which CA it allows?

There is DANE (TLSA) but seemingly only supported by e-mail software and not web browsers.

wood_spirit 5 hours ago

Also on front HN page is https://kaba.cx/posts/20261020_Tor/ on how 300 tor exit nodes were compromised.

In that case it was ssh sessions being intercepted.

But my question is if a tor exit node or a vpn provider or your internet provider has a private key to any CA then can they mitm any normal web traffic even though it will show up as https without warnings in your browser?

basilikum 4 hours ago

Yes.

basilikum 3 minutes ago

Slight specification:

When you're using a modern browser that browser will require any certificate to have a certificate transparency log stamp. Certificate Transparency logs are public append only logs of issued certificates.

So you access example.org. Your ISP uses the compromised CA certificate to create and sign a rogue certificate for example.org. It now has to submit this certificate to a public certificate transparency log and gets back a proof for that. Now it can use that rogue certificate and the proof that it's in a CT log to impersonate example.org and MITM your traffic.

CT does not prevent compromise, but it makes it far easier to detect. If the people behind example.org have a look at the CT logs they will see the rogue certificate there that they did not request. This makes rogue certificates and by extension CA compromise detectable. So your ISP cannot just do this forever in secret.

QuiEgo an hour ago

I assume every state level actor is now quietly trying the first few hundred million private keys possible or so just to see what root ca or issuing ca key pairs match.

navigate8310 3 hours ago

Some websites still using the revoked roots: https://privatebin.net/?cbaaf3a2eae7a2ff#JE3iPaf8Mek91YbFymM...

Particularly concerning is India's UIDAI https://lea.uidai.gov.in/

ben_pfaff 4 hours ago

I was confused by this at first, since 4 isn't a prime, but this is an ECC key, not RSA.

deoxykev 4 hours ago

Seems like the root cause here was human error during EMudhra's key generation ceremony process.

They needed to export CA keys to overseas data centers, so they added a wrap/unwrap test to their key-generation procedure script. Somehow that generated a second set of weak (test?) keys from the CryptoServer Se-Series Gen2 HSM.

Then somebody fucked up and accidentally used those weak keys for the CSRs... then their root and Sectigo signed it. And it shipped.

That's why the private keys are literally one byte.

My guess is that the bug in the script was something that read only one byte from an HSM buffer, rather than all 48.

https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c11

asveikau 3 hours ago

In case anybody else is curious, since the workarounds are only offered for Linux and NetBSD, I was able to fix this on FreeBSD with:

    sudo certctl untrust /usr/share/certs/trusted/emSign_ECC_Root_CA_-_G3.pem

And on OpenBSD by editing /etc/ssl/cert.pem, removing everything starting from the line:

    === /C=IN/OU=emSign PKI/O=eMudhra Technologies Limited/CN=emSign ECC Root CA - G3

Up to but not including this line:

    === /C=IN/OU=emSign PKI/O=eMudhra Technologies Limited/CN=emSign Root CA - G1

denysvitali 2 hours ago

If you can see this, you're affected. Update your system trust store immediately.

https://dv-g3b-emudhra.we-have-a-problem.com/

deathanatos 3 hours ago

Why are the intermediates, and not the root cert, being revoked?

The intermediates seem to be difficult in that the root stores don't seem to have a mechanism for it (… kinda seems like an oversight, at this 20:20 vision point?). If eight of the CA's intermediates are hosed … how many are still "good", and are they good good, or are we just fooling ourselves?

justincormack 2 hours ago

There are 30k websites using certs chained to that root

deathanatos 2 hours ago

I guess the other question I have is just how many intermediates does this one CA have? They've had 8 compromised thus far; that's not the entire lot of them?

LE has two EC intermediates, by comparison.

baobabKoodaa 10 minutes ago

You know the old saying: if you find 8 cockroaches in your house, then a realistic estimate for the total number of cockroaches in your house is exactly 8.

QuiEgo an hour ago

At least Google and Mozilla are revoking the roots too

ghuntley 6 hours ago

Just wrote up a quickie about how Nix + LLMs make patching everything super simple via overlays. Simplest security response I've ever done in my life. http://ghuntley.com/patch

innoying 4 hours ago

I've opened up a PR adding an example certificate generated using these keys to https://badssl.com - https://github.com/chromium/badssl.com/pull/559

rockdoe 5 hours ago

So uh, for various IOT junk like Smart TVs, does that mean the ones that don't get security updates (pretty much all of them) are permanently pwned?

CiPHPerCoder 6 hours ago

If you're responsible for PHP client software, and waiting on composer/ca-bundle to release a new update, be aware that https://github.com/paragonie/certainty has been updated with the 2026-10-20 bundle already.

taherchhabra 2 hours ago

Lot of indian financial institutions run on their products.

TedDoesntTalk 7 hours ago

are there any consequences to eMudhra?

woadwarrior01 6 hours ago

I hadn't heard of the CA before and had to look it up. They're a publicly listed company in India. Markets will price this in.

https://www.nseindia.com/get-quote/equity/EMUDHRA/eMudhra-Li...

itintheory 2 hours ago

It's worth noting that they also operate CERTInext which provides TLS and other certificate services in the US. In particular, CERTInext was chosen to be the CA for InCommon which means it's used by LOTS of universities all over the country. This was a migration from Sectigo that took place this year.

https://incommon.org/certificates

john_strinlai 6 hours ago

they will have to file an incident report, which will be heavily scrutinized, and can possibly (edit: i am leaning towards likely, as more information comes to light) result in their ejection from most/all root stores.

darkwi11ow 6 hours ago

Previously such incidents have lead to the CA business cessation by the involved entities (e.g. Diginotar case). I dont see how it would be different in this case.

navigate8310 5 hours ago

That would damage public perception of the current Indian government, so the administration will do everything to keep the CA afloat.

bawolff 4 hours ago

This is so blatently terrible, its hard to imagine they won't be permenantly removed from the CA ecosystem. I'm pretty sure other CA's have been removed for less.

QuiEgo an hour ago

Google and Mozilla are banning their roots. There’s a good chance if’s a death penalty for the company.