# How Control Plane and Data Plane Interact in Databricks

Let’s take a real-world **ETL workflow** in **Databricks on AWS** to see how the **Control Plane** and **Data Plane** work together.

## **Scenario: Data Processing Pipeline**

You are a **Data Engineer** at an e-commerce company. Your task is to **process customer orders** from an **S3 bucket**, perform **data transformations**, and store the results in a **Delta Table**.

### **Workflow: Steps in Databricks**

1. **You log in to Databricks** and open a notebook.
    
2. **You create a cluster** to process data.
    
3. **You submit a PySpark job** that reads data from **Amazon S3**, cleans it, and writes the output to a **Delta Table** in **AWS Glue Catalog**.
    
4. **Databricks executes the job** and saves the results in the customer’s AWS account.
    
5. **Job completion details are logged** in the Databricks workspace.
    

## How Control Plane and Data Plane Work in Each Step

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1740104566944/50ce8c3d-4568-4d88-88d2-9bf54e80c72b.png align="center")

## Visualization of Control Plane vs. Data Plane in This Scenario

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1740104633003/46e2ea4f-2fa8-42b3-9633-e7dde81d5122.png align="center")

## **Key Takeaways**

* **Security:** Your raw data **never leaves your AWS account** because Databricks only manages job orchestration.
    
* **Scalability:** Databricks spins up EC2 instances dynamically, scaling up/down as needed.
    
* **Efficiency:** The separation allows Databricks to manage metadata while keeping heavy data workloads inside your cloud.
    
* **Integration:** The processed data can be **queried via Databricks SQL, stored in Delta Lake, or moved to another system** (e.g., Redshift, Snowflake).