Databricks Data Engineering Professional Preparation  [draft]

01 - Modeling Data Management Solutions Bronze Ingestion Patterns Ingestion Patterns Singleplex: One-to-One mapping of source datasets to bronze tables. Multiplex: Many-to-One mapping, i.e. many datasets are mapped to one bronze table. Singleplex is the traditional ingestion model where each data source or topic is ingested separately. Singleplex usually works well for batch processing. However, for streaming processing of large datasets, if you have many streaming jobs, one per topic, you will hit the maximum limit of concurrent jobs in your workspace....

April 26, 2023 · 5 min · 1024 words · Eric

Exam Guide - Databricks Certified Data Engineer Professional

General Databricks Certified Data Engineer Professional: link Time allotted to complex exam is 2 hours (120 minutes) Exam fee $200 USD Number of questions 60 Question type: multiple choice questions Passing scores is at least 70% on the overall exam Code Example data manipulation code will be in SQL when possible Structured Streaming code will be in Python Runtime version is DBR 10.4 LTS Practice Exam: link Target Audience Data Engineer, >= 2yoe Advanced, practitioner certification Assess candidates at a level equivalent to two or more years with data engineering with Databricks Expectation Understanding of the Databricks platform and developer tools Ability to build optimised and cleaned data processing pipelines using the Spark and Delta Lake APIs Ability to model data into a Lakehouse using knowledge of general data modeling concepts Ability to make data pipelines secure, reliable, monitored, and tested before deployment Out of Scope The following is not expected of a Professional-level data engineer:...

April 25, 2023 · 6 min · 1267 words · Eric

Snowflake Pro  [draft]

Reference Snowflake: SnowPro Certifications

August 24, 2020 · 1 min · 4 words · Eric

Google Certificated Machine Learning Engineer  [draft] Round out your skills with training

August 17, 2020 · 1 min · 7 words · Eric

AWS SysOps Exam Preparation

Billing and Cost Management User-Defined Cost Allocation Tags User-defined tags are tags that you define, create and apply to resources.After you have create and applied them, you can activate them on the Billing and Cost Management console for cost allocation tracking. The detailed steps are: Log in to the AWS Management Console of the new account Use the Tag Editor to create the new user-defined tags Use the Cost Allocation Tag manager in the payer account to mark the tags as cost allocation tags Certificate Manager (ACM) When you request a public certificate, AWS Certificate Manager (ACM) generates a public/private key pair...

August 8, 2020 · 9 min · 1914 words · Eric