A data engineer created a daily batch ingestion pipeline using a cluster with the latest DBR version to store banking transaction data, and persisted it in a MANAGED DELTA table called prod.gold.all_banking_transactions_daily. The data engineer is constantly receiving complaints from business users who query this table ad hoc through a SQL Serverless Warehouse about poor query performance. Upon analysis, the data engineer identified that these users frequently use high- cardinality columns as filters. The engineer now seeks to implement a data layout optimization technique that is incremental, easy to maintain, and can evolve over time. Which command should the data engineer implement?

• A. Alter the table to use Z-ORDER and implement a periodic OPTIMIZE command.
• B. Alter the table to use Liquid Clustering and implement a periodic OPTIMIZE command.
• C. Alter the table to use Hive-Style Partitions + Z-ORDER and implement a periodic OPTIMIZE command.
• D. Alter the table to use Hive-Style Partitions and implement a periodic OPTIMIZE command.

A data engineer wants to create a cluster using the Databricks CLI for a big ETL pipeline. The cluster should have five workers, one driver of type i3.xlarge, and should use the '14.3.x- scala2.12' runtime. Which command should the data engineer use?

• A. databricks compute create 14.3.x-scala2.12 --num-workers 5 --node-type-id i3.xlarge --cluster- name Data Engineer_cluster
• B. databricks clusters create 14.3.x-scala2.12 --num-workers 5 --node-type-id i3.xlarge --cluster- name DataEngineer_cluster
• C. databricks compute add 14.3.x-scala2.12 --num-workers 5 --node-type-id i3.xlarge --cluster-name Data Engineer_cluster
• D. databricks clusters add 14.3.x-scala2.12 --num-workers 5 --node-type-id i3.xlarge --cluster-name Data Engineer_cluster

A table in the Lakehouse named customer_churn_params is used in churn prediction by the machine learning team. The table contains information about customers derived from a number of upstream sources. Currently, the data engineering team populates this table nightly by overwriting the table with the current valid values derived from upstream data sources.
The churn prediction model used by the ML team is fairly stable in production. The team is only interested in making predictions on records that have changed in the past 24 hours.
Which approach would simplify the identification of these changed records?

• A. Convert the batch job to a Structured Streaming job using the complete output mode; configure a Structured Streaming job to read from the customer_churn_params table and incrementally predict against the churn model.
• B. Replace the current overwrite logic with a merge statement to modify only those records that have changed; write logic to make predictions on the changed records identified by the change data feed.
• C. Apply the churn model to all rows in the customer_churn_params table, but implement logic to perform an upsert into the predictions table that ignores rows where predictions have not changed.
• D. Modify the overwrite logic to include a field populated by calling
  spark.sql.functions.current_timestamp() as data are being written; use this field to identify records written on a particular date.
• E. Calculate the difference between the previous model predictions and the current customer_churn_params on a key identifying unique customers before making new predictions; only make predictions on those customers not in the previous predictions.

A data engineering team uses Databricks Lakehouse Monitoring to track the percent_null metric for a critical column in their Delta table.
The profile metrics table (prod_catalog.prod_schema.customer_data_profile_metrics) stores hourly percent_null values.
The team wants to:
Trigger an alert when the daily average of percent_null exceeds 5% for
three consecutive days.
Ensure that notifications are not spammed during sustained issues.

• A. SELECT AVG(percent_null) AS daily_avg
  FROM prod_catalog.prod_schema.customer_data_profile_metrics
  WHERE window.end >= CURRENT_TIMESTAMP - INTERVAL '3' DAY
  Alert Condition: daily_avg > 5
  Notification Frequency: Each time alert is evaluated
• B. SELECT percent_null
  FROM prod_catalog.prod_schema.customer_data_profile_metrics
  WHERE window.end >= CURRENT_TIMESTAMP - INTERVAL '1' DAY
  Alert Condition: percent_null > 5
  Notification Frequency: At most every 24 hours
• C. SELECT SUM(CASE WHEN percent_null > 5 THEN 1 ELSE 0 END) AS violation_days FROM prod_catalog.prod_schema.customer_data_profile_metrics WHERE window.end >= CURRENT_TIMESTAMP - INTERVAL '3' DAY Alert Condition: violation_days >= 3 Notification Frequency: Just once
• D. WITH daily_avg AS (
  SELECT DATE_TRUNC('DAY', window.end) AS day,
  AVG(percent_null) AS avg_null
  FROM prod_catalog.prod_schema.customer_data_profile_metrics
  GROUP BY DATE_TRUNC('DAY', window.end)
  )
  SELECT day, avg_null
  FROM daily_avg
  ORDER BY day DESC
  LIMIT 3
  Alert Condition: ALL avg_null > 5 for the latest 3 rows
  Notification Frequency: Just once

A platform team lead is responsible for automating the individual teams attribution towards SQL Warehouse usage. The requirement is to identify the SQL warehouse usage at the individual user's level and generate a daily report to be shared with an executive team that includes leaders from all business units. How should the platform lead generate an automated report that can be shared daily?

• A. Restrict users from running any SQL query unless they provide all the query details so that the attribution can be calculated and shared with the executive team.
• B. Use the system tables to capture the audit and billing usage data and create a dashboard with daily refresh schedules and shared with the executive team.
• C. Use the system tables to capture the audit and billing usage data and share the queries with the executive team. This enables the executives to execute the query and see the latest results any time.
• D. Let the users run the SQL query and then directly report the usage to the executives. The ownership of the SQL warehouse usage will be with the individual teams.