Google Accidentally Deletes $135 Billion Pension Fund, Chaos Ensues
77,965
Published 2024-08-06
Chapters:
0:00 - Intro
0:25 - UniSuper Use Case
1:56 - Google VMWare Engine
3:03 - The Outage
6:24 - Sus Google Tool
7:23 - Backups
8:18 - Backup Song
Notes:
1. Some report on an “account being deleted” or a “subscription being deleted” or “multiple (2) clouds being deleted” however under “Scope of the Impact” from the Google postmortem, it is clear that it was a single GCVE cloud in one region stretched over two zones that was deleted (cloud.google.com/vmware-engine/docs/private-clouds…. However that is not very appealing to the less cloud-savvy masses, hence the titles of the articles and this video.
2. The specifics behind why UniSuper needed Google to manually create their GCVE cloud is not publicly known. The RAM limit in the video serves as an example.
3. The external provider with the backups was most likely Azure not AWS as pictured in the video but it's hard to know for sure so w/e (fst.net.au/financial-services-news/unisuper-select…) "Despite the prominence of Google Cloud in its all-cloud transition, Cooper confirmed that the company is committed to a multi-cloud strategy, already utilising Microsoft Azure to operate its digital and integration platforms."
Sources:
cloud.google.com/blog/products/infrastructure/deta…
www.theguardian.com/australia-news/article/2024/ma…
www.itnews.com.au/news/unisuper-points-finger-at-g…
www.fudzilla.com/news/memory-and-storage/59007-goo…
www.reddit.com/r/AusFinance/comments/1cibo5t/unisu…
cloud.google.com/blog/topics/hybrid-cloud/announci…
www.investmentmagazine.com.au/2024/06/an-implausib…
Music Credits:
Thriller Trailer Teaser Tense by Cold Cinema
All Comments (21)
-
“Who TF have multi cloud back up?” “The guy that saved this company”
-
Was so shocked that the company followed the 3-2-1 rule.
-
The outro slaps harder than the Crowdstrike outage
-
Outro had no business being as fire as it was😂😂
-
why reverse a linked list when you can reverse a lifetimes work
-
We all know how this would have been resolved had the deletion been to something worth substantially less than $135B
-
at least they had sensible defaults like deleting everything with no warning after 1 year
-
Nice to know that a company was actually saved from this situation because they had actually followed best practices about IT... sadly rare
-
That outro track is a 10
-
holy shit when the autotune came in i shat my pants, bars
-
Never Ever forget that "the cloud" is just a fancy name for someone else's computer and you're their mercy.
-
To be fair i can easily see how this happens. Its unfortunately very common for engineers to build some tooling for testing or debugging in narrow engineering use cases. Then sales or another department comes along with a problem they need fixed WiTh tHe HigHeSt pRioRiTy and "borrow" the tool "just for this once". A year later the entire department ist misusing the tool or even some bastardized version of it for production use on actual customer data. No one fully understands what the tool does or what it was even meant to do, but it saves some time so everyone just uses it without a second thought.
-
"As a cloud engineer myself, I can confidently say this is complete bullshit, it is 100% UniSuper's fault"
-
Return of the king Can't wait for the future cloudstrike video
-
the rap at the end was hilarious 😂
-
I am actually impressed that they managed to get back up and running in two weeks time after having their entire environment deleted, that's actually a great job by the unisuper it guys, hats off, kudos and respect for a job well done.
-
"When one of disaster strikes" typical insurance scamvider policy don't cover your case on page 25/60 fine print
-
The thing about Australians is that black swans are just regular swans here. Jokes aside, I'm not even surprised that UniSuper had that many redundant backups. Universities take their pension money seriously.
-
bro cooked with that rap, goddamn
-
that outro is fire kevin's kicking off his music career :O
