Databricks ML In Action – Packt Publishing

For this project, no transformations are needed, so our next step with this data will be in Chapter 6. Summary It is critical to understand your data before using it. This chapter highlighted a variety of methods to explore and analyze our data within the Databricks ecosystem. We began by revisiting DLT, this time focusing on how we use a feature called expectations to monitor and improve our data quality. We also introduced Databricks Lakehouse Monitoring as another tool for observing data quality.

Among its many capabilities, Lakehouse Monitoring detects shifts in data distribution and alerts users to anomalies, thus preserving data integrity throughout its life cycle. We used Databricks Assistant to explore data with ad hoc queries written in English and showed why AutoML is an extremely useful tool for data exploration by automatically creating comprehensive data exploration notebooks.

Together, all of these tools create a strong foundation to understand and explore your data. Finally, the chapter delved into Databricks VS and how using it to find similar documents can improve chatbot responses. We have now set the foundation for the next phase of our data journey.

Chapter 5 will focus on how to build upon our bronze-layer data to create rich sets of features for data science and ML projects. Getting to Know Your Data Questions The following questions are meant to solidify key points to remember as well as tie the content back to your own experience: 1. What are some low-code options for data exploration that we discussed in this chapter?

2. When might you use Databricks Assistant for data exploration, and when might you use AutoML’s data profile notebook? 3. How and why would you set expectations on your data? 4. When would you use a regular database versus a vector database? What are some common use cases for vector databases?

Answers After putting thought into the questions, compare your answers to ours: 1. Some low-code data exploration options include using the ydata library, in-cell data profile, Databricks Assistant, and AutoML. 2. Databricks Assistant is useful for data exploration when you have a good idea of the analyses you want to build and you want code assistance. Databricks Assistant is a great way to speed up the coding process or augment your SQL knowledge.

Databricks ML in Action Learn how Databricks supports the entire ML lifecycle end to end from data ingestion to the model deployment Stephanie Rivera Anastasia Prokaieva Amanda Baker Hayley Horn Databricks ML in Action Copyright © 2024 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Group Product Manager: Ali Abidi Publishing Product Manager: Sanjana Gupta Content Development Editor: Priyanka Soam Technical Editor: Kavyashree K S Copy Editor: Safis Editing Project Coordinator: Shambhavi Mishra Proofreader: Priyanka Soam Indexer: Rekha Nair Production Designer: Jyoti Kadam Marketing Coordinator: Nivedita Singh First published: May 2024 Production reference: 2170424 Published by Packt Publishing Ltd.

Grosvenor House 11 St Paul’s Square Birmingham B3 1RB, UK. ISBN 978-1-80056-489-3 www.packtpub.com To the strong women who have come before me, for their sacrifices and for exemplifying the power of determination. To the memory of my grandmother, Hazel Adolph, for being my best friend and cheerleader. – Stephanie Rivera To females who pursued a STEM career and did not give up no matter what obstacles occurred on their way.

Some would say science it’s not for girls, well, prove them wrong. – Anastasia Prokaieva To my mother, Mary Baker. Thank you for showing me what true strength is, for being both a voice of reason and unbridled support, and for believing in me no matter what. – Amanda Baker This is dedicated to the women who inspired me to lead by example, to my mom, Susan Charba, who reminded me that I could be the one who inspires, and to the women still working their way up.

I’ll send the elevator back down. There is plenty of room for us all. – Hayley Horn Contributors About the authors Stephanie Rivera has worked in big data and machine learning since 2011.

This is a short excerpt from the opening of “” by Unknown, quoted for review and introduction purposes. All rights belong to the copyright holders.

Book Information

• Unique ID: 252ff0adda9892e1
• File Extension: .pdf
• File Size: 29,351,420 bytes (27.992 MB)
• Title: –
• Author: Unknown
• ISBN: 9781800564893, 9781098151935
• Pages: 281
• Language: English (en)

Reading & Word Statistics

• Estimated Reading Time: 302.92 minutes
• Total Words: 60,584
• Total Characters: 391,086
• Average Words per Page: 215.6
• Average Characters per Page: 1391.77

Most Frequent Words

data (1047), databricks (659), model (529), figure (473), table (305), feature (296), use (291), project (251), using (229), chapter (228), delta (212), streaming (195), learning (193), tables (190), code (184), training (180), notebook (176), models (175), create (174), features (158), https (157), set (152), com (139), also (139), time (137), function (125), applying (124), see (122), transactions (119), platform (115), catalog (112), example (104), mlflow (102), now (101), new (100), monitoring (99), let’s (97), automl (92), chatbot (92), inference (92), workspace (91), api (90), need (90), building (89), sql (88), used (88), book (87), lakehouse (86), one (85), experiment (85), run (85), creating (84), layer (84), store (83), open (79), favorita (79), want (77), reading (77), schema (77), html (77), dataset (76), next (76), pipeline (76), projects (75), unity (75), section (74), auto (73), production (71), dlt (70), notebooks (70), loader (69), build (69), engineering (68), file (68), transaction (68), first (67), sales (67), bronze (67), python (67), created (67), following (67), files (66), machine (65), stream (65), shown (64), docs (64), get (63), serving (63), vector (62), access (62), search (61), setting (61), please (61), ready (60), look (57), query (56), process (56), information (55), product (55), best (55).

PDF Download