Snowflake’s Data Cloud Summit 2024 is the must-attend event for all the cutting-edge innovations the AI Data Cloud has to offer.
Josh Wright, a Data Engineering Consultant at Data Army, attended the four-day event from 3-6 June in San Francisco and shares his highlights.
Data Governance: Unveiling Snowflake Trail’s Advanced Monitoring and Query Telemetry Features
Our data volumes are rapidly expanding at rates never seen before.
In response, Snowflake has released a plethora of features designed help the community monitor and govern this ever increasing, complex environment.
To maintain control, businesses need to know who is using what data, how are they using it and where did the data come from originally (lineage).
In tandem with the increase in data, our use of User Defined Functions (UDFs), Stored Procedures and Snowpark has also expanded.
Now with Snowflake Trail we will better be able to monitor and debug data pipelines.
The current query profiler is a good start for optimising our larger queries, but with the Query Telemetry that comes with Snowflake Trail we will be able to drill down further to find root causes of errors, timeouts, and long running queries.
Unlocking AI Potential: Exploring Cortex Functions and Snowsight Integration in Snowflake
Of course, there is huge excitement around the newer AI products in Snowflake, both Cortex functions and ones that are integrated into Snowsight itself.
Updates to Cortex AI include the general availability of large language model (LLM) functions that provide serverless access to foundation and embed models.
To enable efficient, easy, and trusted use of generative AI, these new components are coming soon: Cortex Analyst and Cortex Search (both in public preview soon) for natural language interaction with data and document search, Cortex Fine-Tuning (public preview) for secure and effortless LLM customisation, Snowflake AI & ML Studio for no-code AI development (private preview), and Snowflake Cortex Guard (generally available soon) for content safety and model usability.
Cortex Analyst and Cortex Search, which create a chat-bot-like platform for you to question relational and non-relational data respectfully, will be a first port-of-call to play around with.
Snowpark Container Services: Transforming Data Application Deployment and Management
I am probably most excited about the full release of Snowpark Container Services and its integration with Snowflake Apps.
Snowpark Container Services (SPCS) is a fully managed container offering that simplifies the deployment, management, and scaling of containerised services, jobs and functions without requiring any data movement. It integrates with Snowflake features and third-party tools like Docker, enabling teams to focus on building data applications rather than managing infrastructure. SPCS includes Snowflake’s native security, role-based access control (RBAC) support and best practices, to allow seamless operation across regions and clouds. It introduces new object types and constructs to the Snowflake platform, such as images, image registries, compute pools, and more.
I can already see many Snowflake holistic pipelines that I’m keen to create a proof of concept around.
Insights and Innovations from Snowflake Summit 2024
Not only were there the new announcements, I was also able to see how different people in the industry were able to help tackle their unique problems through Snowflake.
The hands-on labs provided an easy-to-follow guide on some of the newer technologies. These followed the quick guides that Snowflake has but were enhanced by thorough explanations into the technologies and core concepts that were required. These made learning and following along with these a lot more enjoyable.
My favourite talk was a behind-the-scenes look into a query. This talk went into more detail about how Snowflake tables are created and partitioned, then how this impacts the distributive computing engine that runs the query. Being able to get a better look at how Snowflake query optimisation thinks and acts is invaluable knowledge at trying to get the best out of our data.
Now that Snowflake Summit 2024 is nearly over, I’m looking forward to getting home and delivering value to clients by applying everything I’ve learned this week solve those tricky problems.
