Just because there isn’t a “novel exploit” doesn’t mean this isn’t a big deal.
Snowflake is susceptible to their SE’s having credentials stolen. These credentials can bypass MFA. And per the article, they have no expiry. That’s strikes one, two, and three.
Snowflake’s security practices lead to a situation where their customers are either required, or at minimum encouraged, to share access to broad datasets with Snowflake employees. That’s strike four.
Yes, there is also issue here that the customers are responsible themselves for not granting too broad access, and that’s on them. But it’s also on Snowflake for not having a better system that doesn’t require this access, or at minimum not having better oversight and control over this transitive access.
Once these accounts are granted access to a customer’s data, they aren’t “demo accounts” anymore. They’re real accounts, with real, very valuable data, and they should be treated as such.
Edit to add: it is worth noting that Snowflake claims the demo account did not have access to customer data and wasn’t the source of the leak, which is in contradiction with what the attackers claim.
Yes, that's also what I think must have happened—missing 2FA. But it seems it's also not mandatory within Snowflake accounts also, from what I understand from the general message.
I've never used Snowflake and assumed that because you push all your data into it, it probably has 2FA enabled by default. Is it optional?
My understanding was that you had to grant storage access (e.g. S3) and compute access (e.g. EC2) from your account to Snowflake, which would then use said resources to perform queries that you issue from their hosted web UI.
In that case it would mean stealing the Snowflake demo account of a SE should not expose your data unless you forgot to revoke their access to your underlying resources.
I always wondered why snowflake doesn't just install a control plane on customers own cloud resources a la databricks. Seems like they'd be able to mitigate a lot of liability that way.
You can definitely bring your own storage (e.g. store your data in your own s3 buckets and integrate it with snowflake) using storage integrations and external tables.
Personally believe this is the right approach as the data resides in a location fully under the company's control. You could ditch snowflake and the data still resides in your s3 buckets for reuse with a another platform (just remove the iam permissions for snowflake).
Yes, federated queries (external tables) are supported but that is a lot slower than ingesting the data into Snowflake's storage and querying it. Since Snowflake's pricing model is based on computation time, querying external tables are usually more costly because of worse performance.
> So the customer data is actually stored on Snowflakes AWS accounts?
Yes.
> Also does that mean every data query to snowflake goes out/in to/from internet at egress/Ingress costs?
Yes. It's covered comprehensively in their docs, along with the caveats.
> What difference does it make what underlying storage / provider it uses then?
"Snowflake does not charge data ingress fees. However, a cloud storage provider might charge a data egress fee for transferring data from the provider to your Snowflake account."
unsaid: "...and you have to pay for that".
Note that when they say 'your Snowflake account' they mean our cloud account which we own, and which we run our workloads in, which we refer to as 'your' snowflake account.
Tangibly speaking, what means is that if you want to check up your billing; you go through snowflake; you can't login to a cloud console and see the actual charges the cloud vendor is charging.
> What difference does it make what underlying storage / provider it uses then?
They pass the specific underlying cloud vendor costs on to you (with, I guess, some markup, though you have no way of know what that is :)
Snowflake usually unloads data to an internal stage bucket in the same region as your snowflake account. If you use an s3 gateway endpoint getting that data is free of egress charges.
At the snowflake size you get custom price lists from cloud operators.
But I think there was also support for peering with client VPCs (or equivalents) which is why they support AWS, Azure, and GCP - you choose the location that is most fitting for linking with your cloud/physical workloads.
This was the case in ~2019 but are you sure this is still true? I think you can “bring your own account” with Snowflake, but I’m honestly not certain because their docs aren’t exactly clear about it…
Having spent a lot of time in the Snowflake ecosystem, a few things to understand.
Snowflake is both a platform and a database. Customers can implement any level of security within the system. For example, when Snowflake introduced OAuth functionality, they put a lot of pressure for us to implement it in our tool. We're small enough and they are a significant partner so we priortised and got it into our platform so that a customer CAN implement it if they want too.
The keyword here is CAN. As a platform with database functionality, Snowflake allows customers to implement any level of security for access. So each instance of Snowflake https://XX123456.us-east-1.snowflakecomputing.com/ security is independent. So let's answer some questions
* "Was this a production breach", No, this was not a production breach. This is not when hackers went into the Azure and were able to elevate credentials and see all accounts.
* "But doesn't Snowflake use MFA", yes and on INTERNAL systems and the ability to connect to the PROD environment of SF it would absolutely have that. However, every customer has the ability to configure a user's credentials for access in their instance https://XX123456.us-east-1.snowflakecomputing.com/. Likely, the SE was granted access to the client's instance through the use of a user name and password.
* "But why not use MFA then Snowflake", this comes back to Okta. You have to create a trust relationship between providers. That's easy when you are only talking about your own employees. They all sit in your same directory. However, to set up a separate trust relationship for another domain, well now we have to get the security team of the client involved. All this so that someone can take a little bit of data and move it into another environment in order to do a demonstration. Not likely.
* "Well that just sounds stupid, my application has SSO enabled." True, but as also a database - Snowflake has to account for entire ecosystem that don't even support SSO-OAuth. For example, two major products that do not support SSO connections to Snowflake: Azure Machine Learning and Zapier. Both only support user name / password connections to Snowflake. So if you were an SE trying to show the value of Snowflake and its integration with AzureML; the ONLY way you could do that would be through user name / password functionality.
The issue in all of this is we are so used to thinking of SaaS as just an application. Databases have to support all sorts of application weirdness. Here is my gut thought on how this went down.
* The SE was working with multiple clients. These clients in an effort to save time and not wanting to have their security team involved and try and create a trust relationship with Snowflake went ahead and created a user name and password for the SE into the client environment vs setting up SSO. Again, this is ONLY controlled by the client.
* The SE at some point had active access to multiple instances of Snowflake to their various clients; with various levels of access depending upon the skill and fastidiousness of the client when they created the account for the Snowflake SE into their account. Again, how the SE is connecting to Snowflake is the same way any other user and application are doing so. I know this for a fact as SEs have to create their own throwaway demo instances https://XX123456.us-east-1.snowflakecomputing.com/. SEs don't have access into the backend of Snowflake.
* At some point this SE had malware installed on to their computer. At this point, this person was cooked and any customer who had given them access to the environment.
As for Hudson Rocks, they can go f themselves for doxing the SE. It was quite trivial to find who this person was and based on the Snowflake announcement they no longer have a job. That person was having an awful week already, now there name is out there. So again, F Hudson for doxing. It would have been easy to blind them out of the exchange (they did the hacker), but did not do so for the victim.
Calling the SE a "victim" is debatable. If you work environment is infected with malware, you screwed up.
"Typically, Lumma has been distributed disguised as cracked or fake popular software like VLC or ChatGPT. Recently though, threat actors have also delivered the malware through emails containing payloads in the form of attachments or links impersonating well-known companies."
I would not be suprised if this was a recruiter-attack-vector.
Snowflake is susceptible to their SE’s having credentials stolen. These credentials can bypass MFA. And per the article, they have no expiry. That’s strikes one, two, and three.
Snowflake’s security practices lead to a situation where their customers are either required, or at minimum encouraged, to share access to broad datasets with Snowflake employees. That’s strike four.
Yes, there is also issue here that the customers are responsible themselves for not granting too broad access, and that’s on them. But it’s also on Snowflake for not having a better system that doesn’t require this access, or at minimum not having better oversight and control over this transitive access.
Once these accounts are granted access to a customer’s data, they aren’t “demo accounts” anymore. They’re real accounts, with real, very valuable data, and they should be treated as such.
Edit to add: it is worth noting that Snowflake claims the demo account did not have access to customer data and wasn’t the source of the leak, which is in contradiction with what the attackers claim.