Muse exposes users' personal and business secrets when they share conversations (arstechnica.com)

407 pointsby gumby4 days ago179 comments

ryandrake 4 days ago

I couldn't be true. You're telling me that Meta, of all companies, would be careless with user data and privacy? Totally unbelievable!

tonyhart7 4 days ago

the worse part its by design for targeted ads

its not incompetent, its designed that way

llljk 4 days ago

The ceo who infamously said this

    Zuck: Yeah so if you ever need info about anyone at Harvard
    Zuck: Just ask.
    Zuck: I have over 4,000 emails, pictures, addresses, SNS
    [Redacted Friend's Name]: What? How'd you manage that one?
    Zuck: People just submitted it.
    Zuck: I don't know why.
    Zuck: They "trust me"
    Zuck: Dumb fucks.

and suspended the instagram account of protest organizer during its peak? https://www.reuters.com/technology/meta-apologizes-after-sus...

anyone surprised?

nomilk 4 days ago

To be fair, this is every tech company, ever, with extremely rare exceptions.

alex1138 4 days ago

To be "fair", Zuckerberg is a mendacious little prick with a history

Did the Google guys ever get caught saying that? Or using their power over the company to unsend messages, oh I guess you can't do that with email, but with Messenger you can...

kube-system 4 days ago

You can absolutely unsend email when the sender and receiver use the same server

alex1138 4 days ago

It doesn't change my point

zaptheimpaler 4 days ago

It's not, there are levels of scumbaggery and Meta is one of the absolute worst. Rigging elections all over the world, privacy violations everywhere, actively scheming to get kids addicted to slop media, privacy destroying recording glasses, the list goes on.

bryanrasmussen 4 days ago

generally the guy with this attitude and willingness to just share with some buddy when asked for it is a mid term hire.

educar 4 days ago

I suspect you are projecting. This is not every tech company, not even remotely close. Maybe the big Silicon Valley companies have misled you into thinking that other software companies don't exist.

nomilk 4 days ago

How many companies can you name whose staff certainly cannot access your data?

I thought I could name one (GitHub), but after checking, they can (e.g. if requested by law enforcement etc).

Therefore, the total I can name is 0.

educar 4 days ago

I didn't think that your comment was about data access. I thought your comment was about the callous remarks by zuck. Most of the companies I work with treat customer data with extreme care and sensitivity and don't throw it around like silicon valley ad tech does.

customguy 4 days ago

How many can you name with something like 'They "trust me" Dumb fucks.'?

The question was not "could the do all shitty things if they were scummy assholes", but "are they all scummy assholes".

nomilk 4 days ago

Totally fair to be annoyed/outraged by this. What I argue is that ~every company has that attitude (Zuck just said the quiet part out loud). Like, we're only kidding ourselves if we think most companies' employees can't peek at our data on whim, or that most companies don't share it with third parties, and probably also (possibly secretly) with government.

IAmBroom 4 days ago

Your argument is predicated on condemning every company without any evidence, based on "I sound cool when I neg".

nomilk 4 days ago

The argument is that most companies have staff who have read access to databases. That is not controversial. I'd estimate 99.5% of companies at least, probably closer to (or above) 99.9%, but I'm not an expert.

Note that this is truthful yet unpopular, which is the opposite of approval-seeking (or trying to "sound cool").

You incorrectly state:

> (the) argument is predicated on condemning every company

yet the argument is that almost every company has at least some staff with read access to databases: "every tech company ... with extremely rare exceptions", "~every" ('~' translates to 'approximately'), and "most companies".

ryandrake 4 days ago

I'm finding out in another parallel thread[1] that you might be right and I might just coincidentally have worked for the few companies that take data protection seriously. I don't think I've worked anywhere since 2009 where rank and file engineers could just go access prod customer data whenever they felt like it, so I started believing that that is outlier behavior done by a few companies with cultures of carelessness like Meta.

It just seems so implausible. As soon as a company is big enough to hire their first General Legal counsel, you'd think that kind of practice would be flagged as highly legally risky and stopped. I'm reminded of the gcreep[2] incident which happened over 15 years ago. What company wants to go in front of a judge and say their policies allowed something like this?

1: https://news.ycombinator.com/item?id=50179136

2: https://www.gawkerarchives.com/5637234/gcreep-google-enginee...

aisenik 4 days ago

It's not even about attitude, the regulatory and legal environment in the US essentially requires it, and where this facility is not openly available it is forced via National Security Letter.

The story of Joe Nacchio and Qwest is instructive. Though the current environment was borne out of the GWoT, the principle of extra-legal state access to all US telecommunications was established long ago.

https://en.wikipedia.org/wiki/Joseph_Nacchio

https://en.wikipedia.org/wiki/2001-2007_NSA_warrantless_surv...

latexr 4 days ago

There is a gulf of difference between accessing your data to comply with the law and accessing your data (and giving it away to friends!) for no reason, while being so nefarious about it that you call people “dumb fucks” and mock them for even believing you’d treat the data properly and then spend years proving that you’re trustworthy, while at the same time showing complete understanding of the harm you’re causing by being yourself so private that you buy the houses next to yours and keep them empty.

nolok 4 days ago

At least the last three I worked or consulted at, though I'm in Europe.

It really goes a long way when the law stop saying "it's not nice if" and starts saying "if you can even if you don't then you're guilty".

keybored 4 days ago

To be fair, <everyone is bad>? Okay, fair is fair. All Big Tech companies or (whatever implicit boundary we are operating with here) are bad.

ryandrake 4 days ago

The "everyone is bad" argument deliberately ignores degree and scale, and just provides cover for the worst. You don't defend a murderer by pointing to someone else and saying "But he punched someone in the face. Both are bad!" See also: "Both sides are bad" political arguments.

wolvoleo 4 days ago

Read Careless People by Sarah Wynn Williams. It was pretty eye opening and I don't think this kind of thing applies to all big tech really.

And that makes sense too. Business culture is very different between IBM, Microsoft, Amazon, Apple. I'm sure we all know the differences

close04 4 days ago

> anyone surprised?

A lot of people actually, even techies. I commented something similar on a different tech website (Ars Technica) highlighting the trustworthiness of the company behind the tool, and that by now people should not be surprised the face eating leopard eats faces. I was met with a lot of implicit defense of Meta and redirecting the attention to "whatabout" Anthropic, OpenAI, etc. (much like sibling reply here makes it an "every tech company" problem). The general agreement in that forum was that the company is not the problem, it's just what AI tooling does.

So either the internet is full of shills, or people are genuinely surprised because they chose ignorance for decades. Or maybe both can be true.

sjdh76 4 days ago

Well talk to cybersec folk, it doesnt matter what tech companies do when you have people who set up hopeless passwords, hand off devices to random strangers, click whatever they feel like despite thousands of warnings is, give the server room entry code to the guy who asked for it etc etc

IAmBroom 4 days ago

Fallacy of appeal to worse problems.

That argument could just as equally validate them ignoring their job altogether and collecting a paycheck from the beach of Aruba.

sjdh76 4 days ago

But its reality of dealing with people who understandably dont move at the same pace the tech industry does. That gap exists and keeps growing. And has consequences.

dam_jackalopes 4 days ago

But that reality has nothing to do with the issue addressed in the article or the defense of Meta that the comment you responded to talks about. Irresponsible or ignorant users didn't cause Meta to ignore security basics in their release and it doesn't explain defending that incompetence.

sjdh76 3 days ago

Google Jeff Horwitz a journo who has covered Meta for many years. He says both are happening. Sometimes Meta is irresponsible and sometime people make mistakes or dont know any better. You reduce it to one dimension at your own risk.

dam_jackalopes 3 days ago

Has he written about this incident? What specifically about user behavior applies to this incident? I'm not reducing anything to anything, you just haven't haven't explained what about user behavior contributed to what the post is about.

sjdh76 3 days ago

duh users are sharing private info with publicly accesible links. The person they share it with can reshare it with whoever they feel like. Obviously the user hasnt thought abt what they are doing. You cant protect everyone from their own ignorance all the time. This is what I meant by a growing gap between what an avg user is expected to know and what they actually know.

dam_jackalopes 3 days ago

"Claroc researchers were able to discover the Muse share links because all of them use the domain name cf.metaaiusercontent.com. Discovery is further made easy by the decision to add the conversation name to the URL, along with a long string of text. Simplified examples of such links are home-sale-budget-9f21c8a3.cf.metaaiusercontent.com, q3-revenue-review-7b1e42f0.cf.metaaiusercontent.com, and loan-application-draft-a4c9e2.cf.metaaiusercontent.com. It’s this last design decision, putting the conversation title in the subdomain rather than in the URL path, that makes public discovery possible."

Yeah, clearly it's the idiot users...

close04 4 days ago

You're saying is that when things are bad, it doesn't matter if someone makes them much worse. I think you're doing it for the sake of argument, and you'll find 1000 caveats why it only applies to what you want it to apply, because there's no possible way you'd go with in in real life. "Sir, you were bleeding anyway so it doesn't matter that we took your kidney".

It matters to the people who don't hand over their phone to a stranger. When someone hands of devices to a random stranger it's an exception and it goes against popular wisdom. When Meta does something hostile towards its users they do it for billions and the practice is normalized.

ryandrake 4 days ago

> So either the internet is full of shills, or people are genuinely surprised because they chose ignorance for decades.

I'd be willing to bet that many, MANY of this stuff in forums are not what we normally think of as shills, as in randos paid by a company to defend them, but regular employees who weirdly take criticism of their employers too personally and lash out to white-knight for them, even though it's not part of their job description.

bix6 4 days ago

How could they not take it personally? They’ve chosen to work for the scum of the earth for some extra gold. To maintain their sanity they have to justify Meta which means stan’ing for Marky Mark.

alex1138 4 days ago

This place included. You can get flagged and downvoted pretty quick.

thatsabadlook 4 days ago

Isn't this how they make money at this point?

ZiiS 4 days ago

Their lifeblood is charging for this data; they shouldn't be careless with it.

Nekorosu 4 days ago

Should and would are two different words.

toddmorey 4 days ago

what's striking to me is this isn't some novel, new-to-the-AI-era vulnerability. We've dealt with the security concerns around discoverable "private" links for 20+ years. There's... so much prior knowledge, art, and discussion.

Twirrim 4 days ago

Given the RIFs and what they've done to engineering culture, I can't help but wonder what else they've done. Like maybe cripple their architectural review processes that might have caught such things?

vikrantrathore 3 days ago

This is what happens when an important position to head the Leading AI lab of the company is given to a startup marketing guy, who pretends to be an Lead AI scientist, whose biggest credential is to sale the same startup at exhorbitant valuation to the the same company. This is the price for embracing better marketers over real innovators.

lenerdenator 4 days ago

I honestly find myself wondering what other sort of things Meta, and Mark Zuckerberg in particular, are careless about.

If you do arrogantly stupid things long enough without a consequence, you'll begin to do them more and in other parts of your life.

What's his tax return look like?

abirch 4 days ago

If you haven't read Careless People, I can't recommend it enough.

claaams 4 days ago

“Dumb fucks” -marky mark zuck

juanre 4 days ago

Muse is a slippery slope from which it will be very hard to climb back. Do not subcontract the managing of your life to Facebook (or to anyone else for that matter). Intelligence has to become a commodity, the infrastructure via which the agents do useful work has to be open source, and the data and learnings and SOPs they produce have to be yours.

moolcool 4 days ago

I see a lot of the use cases for these tools as just ways to counterfeit yourself. It's kind of gross, IMO.

thoughtpeddler 2 days ago

Yes, exactly. I remember when the idea of human biological cloning was brought up some decades ago, and society collectively decided "No". Fast-forward to today and digital cloning is suddenly possible and there's absolutely no questioning it - everyone's on board. I'm surprised.

JumpCrisscross 4 days ago

> But a central design decision for constructing URLs has made the links easy to discover. As a result, researchers from security firm Claroc said they were able to enumerate 6,800 links Muse users created for sharing conversations with others.

To be clear, did the users intended for these conversations to be shared with someone? Meta just created a guessable capability URL?

Or were all Muse users' conversations by default public, just behind an obscure URL?

fxwin 4 days ago

Sounds like it's the former: URLs are guessable/easily enumerable but their contents protected by auth unless they're "published" using the share feature

cloudbonsai 4 days ago

It seems that Meta assigns a unique hostname to each shared conversation. For example:

> fiscal-review-b4a5ff.cf.metaaiusercontent.com

So when someone opens the link, the DNS resolution exposes the unique hostname. The original bug reporter found dozens of these URLs in passive DNS logs.

venzaspa 4 days ago

That is remarkably amateur. I'm not a software engineer by trade nor have worked at a billion dollar company, but knew that to be a bad design within seconds.

kccqzy 4 days ago

It doesn’t strike me as inherently bad. Using a new domain for each conversation is novel, but it certainly enables many techniques. For example what if the shared conversations can contain AI-generated interactive code? Then the browser will enforce isolation (cookies, CORS etc) if you just put the parent domain into the public suffix list. It’s a tradeoff that isn’t clear cut. And it’s definitely more work doing it this way.

cma 4 days ago

You can have all the benefits of both if you use a different magic name for the domain--a random domain, no english from the chat--and then also have the unguessable url path. Expose nothing without valid unguessable stuff in the url.

overfeed 4 days ago

> It doesn’t strike me as inherently bad

It should! DNS is mostly public and therefore shouldn't be used to hold non-public information. Everyone (should) know TLS-protected paths are superior for privacy. The secondary issue is they used low-entropy guessable DNS records at that.

The domain-isolation advantages you mention are easily accomplished by using both a unique, high-entropy subdomain and a long unique path, but rach problem should be addressed individually with explicit goals and not just vibed. Also, aggressively rate-limit probers.

kccqzy 4 days ago

You are just describing more tradeoffs. Look we all agree that there’s insufficient consideration of tradeoffs and there are strictly better solutions.

> DNS is mostly public and therefore shouldn't be used to hold non-public information.

This is just not true if someone is coming from enterprise IT, where DNS requests stay within the corporate network, or if MDM solutions enforce the use of DNS over TLS/HTTPS. I could see it as an honest mistake when someone came to implement this from a different background.

> The secondary issue is they used low-entropy guessable DNS records at that.

The DNS records do contain an unguessable part and they aren’t low entropy.

I get the sense that people on HN just like to dunk on the guy who came up with the design. We should always blame the processes such as insufficient review that lead to this bad outcome. Next time when you make an honest mistake because you didn’t know about something, you should also be treated with respect.

overfeed 3 days ago

> I get the sense that people on HN just like to dunk on the guy who came up with the design

I can't speak for anyone else - but I'm dunking on the organization that failed to catch this - at multiple stages - before it was rolled out.

cmiles74 4 days ago

Bad prompting plus a weak review process?

skohan 4 days ago

I remember back when Facebook was still .edu email addresses only (2004 or 2005 maybe), and my university got access, we discovered you could just grab the user id of another user from their public profile, and paste it in place of your own user id on your own profile, and get access to their profile as if it was your own.

kccqzy 4 days ago

A lot of early 2000s websites are like that. I remember a chat site has something similar: you can view DMs between two other users if you log in with your own account, get a valid cookie, and then substitute parts of the URL.

rahimnathwani 4 days ago

Imagine I have a link of this format:

https://cf.metaaiusercontent.com/q3-revenue-review-7b1e42f0

No one can guess that URL. The q3-revenue-review-7b1e42f0 part is only ever shared with the Meta's server (protected by TLS).

Now imagine the URL is instead:

https://q3-revenue-review-7b1e42f0.cf.metaaiusercontent.com/

The first step for my computer to open that link is to do a DNS query, which immediately shares the entire URL (minus the `https://` and the `/`) with my ISP's DNS server (and anyone sniffing traffic along the path, as DNS queries are usually sent in plain text).

underdeserver 4 days ago

Thanks. It wasn't clear to me what the actual issue was from the headline.

omoikane 4 days ago

Lesson here is that the subdomain names can be queried due to passive DNS records, and putting stuff that is not supposed to be guessable in subdomain names is a bad idea.

fjni 4 days ago

That is a lesson meta engineers shouldn’t have to learn this way.

dgellow 4 days ago

Yes, it’s ok for a startup to make that mistake. For Meta it’s hard to excuse

jjulius 4 days ago

I'd wager there was hardly an engineer in the loop at all. Dollars to donuts this was vibe-coded.

_pdp_ 4 days ago

Someone asked me this morning if I consider Muse a privacy risk. Lol. I do.

But to be honest this is all temporary. Muse, Grok, Dots will have users for the foreseeable future but there will be viable options for self-hosting too.

I am already doing it https://github.com/pdparchitect/noodle/

The Hub powers the entire family inference needs through a single device - 4 users - 8 device about 50 agents - mixture of local models and some subscriptions.

Obviously you cannot get frontier level of intelligence for coding tasks on a single Mac Mini/Studio but most people don't need that. Latest generation of open weight models are fine and in the worst case scenario bring your own regional cloud based inference services that value privacy. There are already plenty of companies in UK and Europe that provide serverless access to GLM, DeepSeek etc.

lmf4lol 4 days ago

We are building a Personal AI Assistant platform in Europe. Oh man. It is FREAKING HARD! We have now 20 paying users and it is amazing what those people do lol. You really can't make it up. The complexity of building such a business is something I've really underestimated. Especially if you allow the harness to evolve. For our 20 users, no sandbox is still the same anymore. Everyone customized it, has crons running, built apps or automations. And then jsut to think that Meta Muse went to a million users in literally no-time. I am not surprised at all. On the other hand. Zuck has literally infinite money to spend. You'd expect better than, right?

EDIT: Because someone downstream rightfully asked if we look at user's sandboxes: Yes! IF they ask us. Our users aren't engineers and often get stuck and even the LLM can't figure it out. THEN - AND ONLY THEN - we jack into the sandbox and help them.

strofocles 4 days ago

literally?

close04 4 days ago

It's a topic that comes up once in a while, usually when it's less relevant to the point being made. The short story [0] is that it can mean both literally or figuratively depending on if you use it literally or figuratively.

> We define literally in two senses:

> 1) in a literal sense or manner : actually

> 2) in effect : virtually

[0] https://www.merriam-webster.com/grammar/misuse-of-literally

Terretta 4 days ago

The truth is in the url.

close04 4 days ago

The actual content goes through the trouble of telling you in every paragraph that your understanding of "truth" is wrong.

Unless all you have at hand (or at head) is a 2 parameter small language model, you have no excuse to go no further than the URL.

jonathanstrange 4 days ago

Good to know how much you're "looking" closely after everything your customers are doing. The name of your company is in your profile, right?

lmf4lol 4 days ago

Yes. We are Bluehouse. The homepage still sucks but we are working on a nice one right now without AI Slop :-)

yellow_lead 4 days ago

Crazy to hear. So you look in user's sandboxes, does this comply with GDPR and other regulations?

timthelion 4 days ago

Actually, for the purpose of providing the service its fine to look. They just aren't allowed to use what they see anywhere else. Just like your doctor hears about your health problems but can't sell your medical records. Whats with the nasty toxicity?

not-kinsale-joe 4 days ago

Can IT support in EU remote connect to view a users PC? or does that violate GDPR and other regulations?

baq 4 days ago

If the user agrees it’s fine, GDPR is really not the biggest concern in this case since the user should express consent anyway.

lmf4lol 4 days ago

We only look IF they ask us. Our users aren't engineers and often get stuck and even the LLM can't figure it out. Then - AND ONLY THEN - we jack into the sandbox and help them.

For instance, one customer has 15(!) cronjobs. And they all used LLM calls to do stuff. So they ran out of tokens super quick and asked us for help because they didn' tknow what to do. Its not an engineering company, the owner has a sales background. So we went in and wrote scripts for them so that the cron calls don't need to use the LLM anymore.

sksksk 4 days ago

That’s interesting, so in that scenario, you fixed it for that customer.

How do you then ensure it’s fixed for customers in the future? Do you add instructions to agents to write scripts where possible?

lmf4lol 4 days ago

Yes. Or we make a skill of it and offer it via our skills marketplace. Depends really on what the fix was. Sometimes its really a one-off issue, and sometimes it makes sense to roll the new "insights" out to everyone.

sksksk 4 days ago

Thanks!

Another question I have, is from a product perspective, how do you decide how much of the internals you show to users (like token usage).

I always thought that seems like (from a non technical users POV), like a implementation detail; and quite hard to map to actual usage. For example, one request might use lots of tokens, and another will barely use any; but it can be difficult for a non technical user to undertand the difference.

lmf4lol 4 days ago

intuition and a lot of talking with our customers :-)

newswasboring 4 days ago

I am not a user of your service, but a "system diagnostics" skill is something I have been meaning to develop for my hermes instance. Things like checking if I have left any running servers, cleaning up consumed linux ISOs from jellyfin, check for any repeated tool failures etc. Also its fun to say "run system diagnostics".

m4rtink 4 days ago

Even just the possibility anyone other than the user can "take a look" makes a service like this unfit for purpose.

weird-eye-issue 4 days ago

You do realize basically every service does this right?

m4rtink 4 days ago

Sure, which is why I think if one wants to run any agents with sensitive data, they need to be fully self hosted so that operators can't look into your data. Anything else is insane and will be used against you.

lmf4lol 4 days ago

Do you feel the same way about emails, cloud drives or SaaS like Salesforce or Jira?

weird-eye-issue 4 days ago

> will be used against you

I mean I'm all for privacy and security but this statement right here just comes across as pretty paranoid

threecheese 4 days ago

- A woman using Claude for her diary just had her information used against her. - A nytimes civil suit against OpenAI just had user data handed over to lawyers. - US federal agencies are buying ad marketplace data and joining it with other commercial and government data to extra-judicially identify persons

We are giving an unprecedented amount of personal data to these tools at the same time that the technology is being leveraged against people at an unprecedented scope and scale. No US legal framework exists to protect you against this. You should take this risk seriously.

ryandrake 4 days ago

At the company I work for, a human being gaining access to “look” at a user’s actual prod data (anything that might contain PII) is super rare and very, very difficult. And I’m not even in Europe. At least a dozen people need to approve the access, including people very senior in leadership, you need to describe exactly what data you need to see, what the business purpose is, what steps you are taking to anonymize the data (can you live with a redacted view?), exactly how long you need that access, how you are going to destroy any copies after that access is finished, and so on. This employee access, IF approved, gets logged and can be scrutinized later. And I have absolutely no doubt if you misused that access it would be noticed and you’d be fired (it has happened). I’m pretty confident that nobody at my company is going on the internet and posting “OMFG LOL our users have such crazy data u should see it LOL.”

I always assumed most SeriousBusiness companies over a certain size, not run by 20 year olds, operate similarly.

m4rtink 4 days ago

Yeah, that is the correct behavior what I would expect in a business setting - its a basic decency, not to mention customers switching to competition if they find out their data is not protected.

rcxdude 4 days ago

What do you do when you're providing tech support to a customer about an issue that is related to their data? Obviously there should be some process in place but a customer going 'hey could you look at this' is a perfectly valid reason to, you know, look at it.

And it's not particularly unusual to discuss vague details of your customers anonymously. Even in the medical field, with very strict privacy rules, doctors will often publicly discuss encounters they have had with patients, and they only cross the line when they identify the patient.

ryandrake 4 days ago

That’s a good question, I honestly have no idea how customer support works. My guess is they have access to heavily audited systems that automate away these approvals, are limited to tools that can delete/reset but can’t read, and/or are given a filtered/redacted window to the user data they do actually need to see with human eyes.

weird-eye-issue 4 days ago

> anything that might contain PII

Sounds like your software might require HIPAA compliance in which case everything that you just said makes sense and is likely required in that context. But for the average SaaS that doesn't need HIPAA compliance it doesn't work like that.

csallen 4 days ago

If that's all true then it's almost certainly because you work at a large, mature, bureaucratic, unusually privacy-focused company. Verrrrry few companies work like this when they're small and new. I'd round down to ~0%. And I would bet that, if your company was ever a small startup, it didn't work like this back then, either.

Also, this idea that it's bad or weird to anonymously discuss customers is strange. I've known cops, doctors, dentists, and all sorts of other people who routinely share stories about weird things that happen with people and customers at work, anonymously. This is the norm in the world and always has been.

It's also the norm for the companies who own your data to look at it, when asked to. What do you think customer support even is?

It's only really a small corner of the tech world that treats these things as strange and sacrilege.

customguy 4 days ago

> For instance, one customer has 15(!) cronjobs.

But they didn't ask US for help. We didn't need to know a customer of yours is running 15 cronjobs. Even if this factoid itself may be harmless, it's a good thing to practice on how you talk about these things. I'm not saying you're doing anything wrong, I don't know that, but I totally get why how you talk about it rubs some people (including me) the wrong way.

> Its not an engineering company, the owner has a sales background.

Again, it's good if you have a close relationship with your customers, but we don't, and they don't have one with us.

lmf4lol 4 days ago

Thanks for the constructive feedback. I think you may be right about this. It thought it was anonymous enough in the way I phrased it. Just wanted to be honest.

m4rtink 4 days ago

So dump all my secrets, then have people running the service causually spy on me & brag about it on the Internet ? Marvelous, 10 out of 10!

bcvvvy 4 days ago

Yeah doesn’t look good.

To be fair agentic platforms are a shitty idea from a security perspective anyway. Everyone just threw that out of the window because shiny. Their funeral.

sonicvroooom 4 days ago

but it's fun to use, makes up for your lack of time or focus to build certain skills in the past and you live 30, 40, 50, 60 more years ...

what's the worst that can happen?

Hackers trolling you? Stealing and deforming your projects? Hackers sharing personal info about you and your family and/or kids with the trashiest of people from your past or with your local authorities and wannabe matadors? All without your knowledge but you notice and get pseudo-paranoid and raise your security levels although it is technically and practically too fucking late?

Damn, how much these companies Anthropic, OpenAI, Microsoft, Meta, and ISPs and their hardware suppliers and all those parties and mayors and cities care about their paying customers and users and the people, workers and their children ...

Let's build more data centers and make AI infer smarter & faster. Better margin, better life.

Addendum: We have entered the era where companies carry more responsibility--if not all of it in tandem--for their users' digital security and privacy than the users themselves, especially those for whom the RAM et al prices are a tad bit much but who still pump considerable amounts of time, data and money into various services.

ForHackernews 4 days ago

... every cloud platform keeps an eye on what their users are doing. If you don't like it, don't use someone else's computer.

expedition32 4 days ago

Microsoft is currently desperate to prove that they are not an arm of the CIA.

Telemakhos 4 days ago

Search engines (Google was funded by In-Q-Tel) aimed at searching and reading publicly accessible data. Then they moved to the "deep web" like email with Gmail, searching and reading less publicly accessible data.

Social media convinced people to publish on the web details of their personal life that they never would have voluntarily handed over to federal agents: political views, personal relationships, pictures of everywhere they've been, graphs of associates.

AI gets people to interactively divulge what they're working on and even to upload the files. It's the next step on a path from a past where intelligence agencies had to spy on people to a future where people pay for the privilege of volunteering everything to the intelligence agencies.

edmundsauto 3 days ago

In-Q-Tel and Sony put $527k combined into Keyhole, a mapping startup that google acquired in 2004 (1 year after the original investment). In Nov 2005, Sony and IQT sold google for about $2.2M, by then google had IPOd for $1.6 billion.

I wasn’t familiar with the details but that’s not a very convincing connection to derive anything from. Is there more evidence to the story? Seems like a small amount of money and they briefly crossed organizational paths for about a year, over 20 years ago.

reliabilityguy 4 days ago

Are you saying that AWS is peeking inside the VM you’ve deployed?

ForHackernews 4 days ago

Sure, why not? Unless you've encrypted the thing with keys you (not AWS) control there's absolutely nothing stopping them from doing so -- and even if you do have the drive encrypted at rest, they still have access via the virtualization layer while it's running.

Amazon have very expensive lawyers, I'm sure that their T&Cs are flexible and woolly enough to include some reasonable justification for doing whatever they please with your VM.

"in order to secure our service against abuse..."

"in response to a court order..."

"...in order to provide better service, we reserve the right to."

schwank 4 days ago

Or you could actually read the T&C before saying things like "I'm sure..." and being totally wrong.

https://aws.amazon.com/service-terms/

96. AWS Nitro System

AWS personnel do not have access to Your Content on AWS Nitro System EC2 instances. There are no technical means or APIs available to AWS personnel to read, copy, extract, modify, or otherwise access Your Content on an AWS Nitro System EC2 instance or encrypted-EBS volume attached to an AWS Nitro System EC2 instance. Access to AWS Nitro System EC2 instance APIs – which enable AWS personnel to operate the system without access to Your Content - is always logged, and always requires authentication and authorization. /quote

This has been third party validated (there's a blog) and is part of the contract terms. Which you would find if you did any research before posting. I'll let you figure out how you don't understand KMS either.

ForHackernews 4 days ago

"AWS Nitro" is their new high-security platform, not generic EC2. The obvious implication is that they can "read, copy, extract, modify, or otherwise access Your Content" on non-Nitro instances.

Good for them, if it's really the case that they can't read Nitro EC2 instances. Of course, that still leaves S3, RDS, DynamoDB, Elasticache...

reliabilityguy 2 days ago

So, let’s get back to the original question of VMs being accessible by AWS employees. Can you explain how it happens?

ForHackernews 4 days ago

https://www.wsj.com/articles/amazon-scooped-up-data-from-its...

reliabilityguy 2 days ago

How’s this article about AWS?

bossyTeacher 4 days ago

Why would someone use you rather than using a product from a frontier lab? Surely, you can't be cheaper than them unless your model is an open-source model? Anthropic already offers the equivalent of a cron job and app building tools (along with state management).

DelightOne 4 days ago

Many people don't wanna depend on US products. If US switches them off, you are gone. Not even talking about privacy of US companies. Nor talking about guardrails that let you not do things you need to do.

Sharlin 4 days ago

Because recent events have abundantly shown that dependence of any US SaaS is a big liability.

dgellow 4 days ago

The US is openly supporting a country that regularly threatens to nuke part of Europe and already invaded multiple of their European neighbors

piterrro 4 days ago

I dont get why they bother with DNS and subdomains when they could generate a unique url. This would eliminate the possibility of easy reverse dns lookup.

ulfw 4 days ago

For the life of me I will never understand why people are so eager to use these things

prasadjoglekar 4 days ago

For the same reason people buy shiny new cars, bikes, keyboards....you name it. The novelty and sometimes bragging rights in the circles they inhabit.

ulfw 4 days ago

Yea I think that's about as far as the usefulness goes for personal use

sksksk 4 days ago

I run an personal AI agent locally (hermes), and use a combination of local models and whatever model I can get for a reasonable price (usually deepseek).

After using it for a couple months, I'm converted. They're so good, the more you use it, the more useful it is; you internalise more of what it could be used for. It runs so many boring parts of my life, leaving me with more time to spend with my family.

I liken it to release of the first iPhone; it didn't invent any new technology, but packaged different bits of tech together in a way that was consumer friendly. I think we'll look back at OpenClaw/Hermes/Muse/Instinct in the same way.

I do worry about using purely cloud based offerings like Muse, I reckon there is a market for someone to package an consumer AI agent that lives on your home network.

sreejithr 4 days ago

That's pretty much what Openpod.live is doing

ad_fontes 4 days ago

Whenever this comes up, all I ever see are hand-wavy responses like this. "It does everything", "it changes your life." But how exactly? Because I can only see this being true for the terminally online.

sksksk 4 days ago

That's fair, I'll give a couple of example of things that I've used it for...

I built an app (in codex), that goes through my emails, pulls out any receipts, and adds the purchases to a sqlite database. I do all my grocery shopping online (through my supermarket), so our weekly shop is stored in the database.

Hermes uses that to predict what items will need restocking (based on how often I buy it, and when I last bought it).

Every week, it logs into the supermarket website on my behalf, books a delivery slot, and adds all the items that need restocking to the shop.

Throughout the week, if I run out of something, I take a photo of the barcode and send it to hermes, it adds it to the shop for me.

I built a similar app for travel; if I have any upcoming travel, hermes will send me updates (delays, platform changes etc...). I do a lot of train travel for work, if my train is delayed, it will automatically apply for the compensation on my behalf.

It reads my emails, and pulls any actionable items out, and adds it to an Asana list. I can either do the tasks myself, or assign it to Hermes and it will do it for me. As an example, I had an task to book swimming lessons for my son, hermes monitors the swimming lesson availability and automatically booked it when a suitable one becomes available.

Shopping is a good use case, I have a clock that needed a new battery, it wasn't a standard battery, and required a screwdriver to open the compartment. I took a photo of the clock, sent it to hermes, and it found out what battery took and ordered it of Amazon for me.

I wanted to buy some smart thermostats for our radiators, but wanted to be compatiable with our current smart heating system. Hermes is connected to my home assistant, so it figured out what heating system we have and found a compatiable set of eBay (this is slightly more involved than it sounds, as there is a new version of the heating system with a new set of completely incompatiable accesories).

I don't think "it does everything" or that it has "changed my life", but all of those examples are chores, things that I don't particularly enjoy doing; and if something can take it off my plate, then it's useful for me.

gherkinnn 4 days ago

Your thermostat and non-standard battery are examples of LLMs being helpful.

As for the rest, that is a lot of tokens burned to keep a fridge stocked.

sksksk 4 days ago

It doesn't burn that many tokens, as most of the code is deterministic.

Receipt emails are in a predictable format, so codex built scripts to parse the emails. If there is a receipt email it doesn't recognise, it will use an LLM to build a parser for that kind of email once.

The agent also "self improves", so after asking once to get items to restock, it wrote a skill with the query it used in, so it doesn't burn tokens figuring it out again.

I was impressed that after it logged into my supermarket account, it inspected the network traffic and discovered a GraphQL API, and used that (instead of screen scraping); much more economic.

I've averaged 2.6 cents a day in token costs over the last month.

ryandrake 4 days ago

Thanks for your detailed answers. Although none of what you posted even remotely interests me as something I’d be interested in doing on a computer, it is absolutely fascinating to see details about how others are using LLMs and turning their computers into these “assistants.”

sksksk 4 days ago

Yeah it’s not for everyone, it reminds me of the automation xkcd comic [1]

For me my motivation is that I’m working 9-5, my children go to bed at 8pm, and I don’t have my phone on me so I can spend time with them.

Once they’re asleep, I’ve got 2 hours to spend with my partner before bed. I’d rather actually spend time with her rather than checking if we have enough kitchen roll.

[1] https://xkcd.com/1319/

gherkinnn 2 days ago

Thanks for the details. Maybe I remain too conservative.

convict 4 days ago

When i run out of something to buy from the store, i type it in the notes app on my phone

sksksk 4 days ago

When I need to change the channel, I just get up and walk over to the television.

latexr 4 days ago

I understand what you’re getting at, but that example doesn’t bode well for you. The remote control increased convenience at the cost of making us more sedentary and lowering our attention spans. It was not a good trade.

331c8c71 4 days ago

I was thinking that I actually use grocery shopping as a recovery activity. I could cut it out but don't feel the need that much. It's not that I feel I'd do something incredible instead (which requires energy and/or recovery time of its own)...

ulfw 4 days ago

Oh I convinced now!

I'll use personal agents for my laundry now. Thanks mate for the samples given

sksksk 4 days ago

no need to be rude, see my reply to a sibling comment for examples

dgellow 4 days ago

If the AI companies would actually automate laundry, managing my dishwasher, clean my house I would be so willing to praise them as the best thing ever created. Instead we are getting the most over engineered automation stack ever conceptualized. And it’s by design not reliable

SoftTalker 4 days ago

> automate laundry, managing my dishwasher, clean my house

You can actually hire a person to do these things and it will likely be cheaper than any technological solution.

ctolsen 4 days ago

I am the opposite. I work with LLMs all the time but don’t use them much at all personally. Not for lack of trying, I’ve tried several times with different frameworks and I just can’t find a use case that is remotely worth it.

skohan 4 days ago

I think the promise of personal AI agents is incredible, and they can already be super useful. But we haven't even begun to figure out the safety side for a mass audience.

Even as an experienced developer, sandboxing an agent is not trivial. The idea of my aunt, who has her passwords on a post-it-note on her monitor and forwards me scam emails using these things is terrifying.

sourcecodeplz 4 days ago

i understand the adoption for these things is off the charts.

usage-time growing fast, with tasks and cron jobs.

in the last spacex earnings report (few hours ago), Elon said he thinks their grok-bot will next year make up most of their AI inference usage across their datacenters.

basically he tried to imply these bots/personal-agents get lots of usage once users start customizing them. and it is bringing in a lot of money in API usage.

educar 4 days ago

You cannot understand it because you are possibly thinking of it from a utility perspective? People just do things because it's cool or keep themselves entertained. Mostly to keep themselves entertained.

latexr 4 days ago

Because a lot of money is being poured into fear mongering campaigns that if you don’t you will *puts flashlight below chin* “be left behind”.

No one ever answers why that’s bad, though, or behind what. Turns out it’s behind in the procession walking towards the cliff. Maybe not such a bad idea to stay behind on that one.

mbesto 4 days ago

My personal assistant is treating exactly like a personal assistant - which means it has its own email, phone number, etc. It runs on OpenClaw.

My biggest use case is to coordinate groups. For example I tend to do big travel trips with like 4-15 people. Having an assistant coordinate group payments ("split this bill and get everyone's venmo accounts"), signups ("who wants to do horseback riding?"), takes all of "work" out of vacations.

I will NOT let it touch anything that it could unintentionally (or intentionally) leak sensitive personal info. So, anything that has a credit card, bank account, etc. is pretty much offlimits.

thoughtpeddler 2 days ago

I believe that in the fullness of time we will see that by the mid/late-2020s our society was completely information-saturated/overloaded, and hence why turning to these personal AI agents was seen by most as an easy decision (if not downright necessary to gain a feeling of control again over their lives).

Most peers (Millennial knowledge workers) I talk to these days constantly discuss the feeling of "being overwhelmed", and I suspect a lot of it is just the info-sphere.

Neil Postman predicted some of this many decades ago in "Informing Ourselves to Death", where he argued that increasing the speed, volume, and accessibility of information would not necessarily improve society (i.e. information without context/judgment and institutional scaffolding can become socially destructive).

I really feel like we live in that world, and hence the rise of these AI agents as a potential "solution" to that.

viccis 4 days ago

Muse is the most obviously doomed claw-style bot because you already need to segment those things from ANY personal info, and Meta rolls into the room as the journal that will tell everyone what you write in it and says "trust me"

Part of a huge problem with FAANG companies. You can't be Meta and just ratfuck your users for decades and then ask them to trust you, just like you can't be Amazon and turn your storefront into Temu-lite and expect people to want to order products sight unseen with your Echo shit.

dgellow 4 days ago

Actually, they can. And given their distribution channels they can even pull it off. You and me will obviously never use something like Muse from Meta, but they are the best positioned to sell that product to teenagers, young adults who don’t care about privacy and want to feel they are using the trendy stuff. Just look at their mascot, the thing is made to be targeted at younger people. And they will definitely release it as a tamagochi people will cherish as if it was a pet.

Terrible company, doing terrible things for society. But I wouldn’t bet against their ability to sell something to their audience

viccis 4 days ago

Interesting because afaik young people have no interest in Facebook at all. My expectation was that old people, Meta's primary victims, would be the main users. They love mascots like that (see: Minions). They're also likely to be more willing to pay a small subscription fee, which Meta has mentioned they plan on adding.

dgellow 3 days ago

Young people are using Instagram and WhatsApp non stop. Thread is also growing fast, though the demographic isn’t too clear

hamburglar 4 days ago

I am baffled by all these people giving muse their personal info. I tried muse out. Didn’t install anything locally and didn’t give it any personal info beyond a fresh meta identity. I just poked at the free vm for a while. There’s no way in hell I’m giving it access to anything sensitive.

SoftTalker 4 days ago

Oh come on! How will you possibly proactively handle all the very difficult tasks in your life such as booking hotels, ordering groceries, or making dinner reservations?

motbus3 4 days ago

My honest view is that it is taking too long for them to be made legally responsible for these problems.

There is no point on saying "there is no guarantee software" as it is supposed to be "maybe it does not work" and not "it might make your life terrible"

kwarsh 4 days ago

Anyone who jumped on the personal AI agent train and gave their Muse/Instinct/Poke/Grokbot access to everything they own deserves whatever happens to them

SamInTheShell 3 days ago

So grandma just trying to do her taxes doesn’t matter anymore, huh?

And when you’re dumb enough to trust a giant corporation, and this happens to you, we shouldn’t care about you either, right?

kristopolous 4 days ago

This is news? Didn't we all just presume that was what was happening? I mean they're a data mining company.

latexr 4 days ago

> This is news?

Yes, it is. Just because you expected it doesn’t mean it shouldn’t be reported.

> Didn't we all just presume that was what was happening? I mean they're a data mining company.

I know that, and you know that. Many people obviously still use their products and don’t understand that.

kazinator 4 days ago

Separately from the issue of the random component being in the domain name, is it true that it's just a synopsis of the chat subject plus 8 hex digits (32 bit integer), like the examples in the article? Or are those examples shortened for brevity?

bicepjai 2 days ago

What did anyone think was going to happen? I bet 3 dollars it’s vibe coded.

eviks 4 days ago

> Muse, which Meta CEO Mark Zuckerberg said last month was “built from the ground up for privacy and security,”

> reported the findings through Meta’s bug bounty program on October 8 and also tried reaching people at Meta on LinkedIn, but got no reply either way.

Checks out

> The ease in discovering share links further justifies asking if Muse is worthy of the access it requires.

That's all you could justify?

adishiktensward 4 days ago

Zuck never beating the allegations

hmokiguess 4 days ago

Isn't it OpenClaw under the hood? Not surprised.

I like Instinct.

giorgiogamba 4 days ago

There was a feature requirement for this

Danox 4 days ago

What did you expect? :)

vektormemory 4 days ago

If only there was a company that didn't use ads, protected people's privacy, didn't sell their data, and had local data sovereignty?

crorella 4 days ago

oops! This is totally unexpected coming from Meta.

postatic 4 days ago

This is a feature no?

johnnienaked 4 days ago

Comedy of errors

Razengan 4 days ago

I mean, not to victim blame, but anyone willingly using any Facebook product at this point should know what they're signing up for.

wayvey 4 days ago

Unfortunately people are still not that smart or aware of the risks. They should be, but that's not the reality yet.

pferde 4 days ago

It's worse. Even if you talk to them, explain in detail why and how Meta and their products are evil, they will just nod, frown, go "yeah, that's serious, someone should do something", and then they will turn around, check their Whatsapp messages and post a story on Instagram about their lunch.

bwg2000 4 days ago

I went to watch the Social Reckoning (new sequal to the social network), which was somewhat thought provoking. After it finished, I get up to leave and see a couple in front of me immediately start scrolling through facebook and not speaking to each other - made me chuckle!

pferde 3 days ago

Maybe they started deleting everything, before disabling their accounts? One can but hope. :)

Razengan 4 days ago

Can some post ads on Facebook telling people to get off Facebook?

gherkinnn 4 days ago

Watch those ads get pulled. And then see actual scams being promoted.

alex1138 4 days ago

Network effects matter and no you can't just 'email/call them'. Not everyone automatically has those. You can get it, but something like FB, loose connections, regaining contact (you literally don't have their email/phone numbers and don't tell me 'oh but you lost touch for a reason'), is a nice thing if its CEO wasn't who he is

touristtam 4 days ago

Sure but for most people they will not understand the consequences until it is hammered by constant notifications. And those will not happened until such law are mandating for the users to be notification left, right and center. After all Facebook is one of the original promoter of dark patterns in UX/UI.