General Databricks Certified Data Engineer Professional: link Time allotted to complex exam is 2 hours (120 minutes) Exam fee $200 USD Number of questions 60 Question type: multiple choice questions Passing scores is at least 70% on the overall exam Code Example data manipulation code will be in SQL when possible Structured Streaming code will be in Python Runtime version is DBR 10.4 LTS Practice Exam: link Target Audience Data Engineer, >= 2yoe Advanced, practitioner certification Assess candidates at a level equivalent to two or more years with data engineering with Databricks Expectation Understanding of the Databricks platform and developer tools Ability to build optimised and cleaned data processing pipelines using the Spark and Delta Lake APIs Ability to model data into a Lakehouse using knowledge of general data modeling concepts Ability to make data pipelines secure, reliable, monitored, and tested before deployment Out of Scope The following is not expected of a Professional-level data engineer:...

User Scenario Prompt for Cover Letter Write a conversational cover letter for a job application as a [position] at [company] using my resume below as a reference. Draft a persuasive cover letter in 150 words or less highlighting my qualifications and enthusiasm for the [position] at [company] using my resume achievements below. Create a compelling cover letter that explain why I am the best fit for the [position] at [company]. Write the letter using the StoryBrand Framework....

Table Partitioning Table Cloning Table Overwriting Performance Optimisation OPTIMIZE VACUUM Delta Live Table Create a Silver Table from a Bronze Table Data Ingestion Multiple Data Source Single Data Source Data Quality Quality Enforcement with CONSTRAINT General GLOBAL TEMP VIEW CREATE TABLE USING Auto Loader VS COPY INTO SCD Type 2 Table Change Data Capture (CDC) feed Change Data Feed (CDF) Dynamic View Column level control Row level control Structured Streaming Streaming query to handle late-arriving data Streaming query to calculate business-level aggregation for each non-overlapping five-minute interval....

Delta Lake builds upon standard data formats. Delta Lake table gets stored on storage in one or more data files in Apache Parquet format, along with transaction logs in JSON format. Reference Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics Michael Armburst (@databricks) Ali Ghodsi (@databricks, @uc berkeley) Reynold Xin (@databricks) Matei Zaharia (@databricks, @stanford) Michael Armburst: Boston Spark Meetup @ Wayfair / Delta Lake: Open Source Reliability and Quality for Data Lakes Delta Lake Inside YouTube: Understanding Delta File Logs - The Heart of the Delta Lake

Documentation Azure Databricks documentation, link Databricks documentation, link Data objects in Databricks, link SQL Language Reference, link Databricks Alphabetical list of built-in functions, link Data Engineering With Databricks, GitHub Tools Databricks Utilities, link Unzip dbc files, GitHub Tutorials Data Engineering With Databricks, GitHub Databricks Training tutorials Databricks: Databricks Training, YouTube Databricks on the AWS Cloud Quick Start Reference Deployment dbt: Configure Databricks for dbt Cloud