One platform, learned properly
Not a whistle-stop tour of nine tools. Azure and Databricks in depth, which is what the job ads keep asking for.
Data Engineering Training
Learn the Azure Databricks stack a real data team runs: SQL, Spark, Delta Lake, Unity Catalog, Data Factory and Power BI, on one lakehouse you keep building.
About this course
What Data Engineering at Intellea actually covers
Data engineering is the job everybody else is waiting on. No dashboard gets built, no model gets trained and no report gets written until somebody has moved the data, cleaned it, modelled it and made it trustworthy. This track teaches that job on the stack Indian enterprises are hiring for hardest right now: Azure and Databricks.
You start with SQL, from SELECT. Then an Azure account from scratch, then a Databricks workspace and a cluster you size yourself. After that you stay in Spark for a long stretch, because reading and writing data and transforming DataFrames is what the job actually is on most days. Delta Lake, the medallion architecture, slowly changing dimensions and Unity Catalog turn that pile of notebooks into a governed lakehouse. Data Factory and Lakeflow Jobs put it on a schedule. Power BI puts it in front of the business.
The last stretch is what separates an engineer from someone running notebooks by hand: finding out why a job that took four minutes now takes forty, keeping notebook code in Git without the merge conflicts turning ugly, and shipping a change through CI/CD instead of clicking Run in production.
SQL starts at SELECT
The first module is SQL practice from the ground up. You do not need to arrive knowing what a join is.
Spark is most of the course
Compute, Spark SQL, tables and views, reading and writing, and two full modules of DataFrame transformations. That is where a data engineer spends the week.
It ends at the dashboard
Most data engineering courses stop at the table. This one carries on into Power BI, so you can walk someone through the whole chain from raw file to a report a manager would open.
Outcomes
By the end, you can do this
- Write SQL confidently: joins, aggregations, CTEs and window functions, including the patterns that come up in every data engineering interview.
- Stand up an Azure account, a Databricks workspace and a cluster, and choose cluster settings without quietly burning credit overnight.
- Read and write data in Spark across CSV, JSON, Parquet and Delta, including awkward schemas, corrupt rows and partitioned folders.
- Transform DataFrames properly: joins, aggregations, windows, functions, and the difference between a narrow and a wide transformation.
- Build a medallion architecture where bronze stays raw, silver gets cleaned and gold is modelled for the business.
- Run Delta Lake in production: ACID transactions, time travel, MERGE, OPTIMIZE and VACUUM.
- Implement type 1 and type 2 slowly changing dimensions so history is preserved instead of overwritten.
- Build ingestion pipelines in Azure Data Factory with parameters, triggers, retries and monitoring.
- Schedule and operate the whole thing with Lakeflow Jobs, including dependencies, retries and alerts on failure.
- Govern the platform in Unity Catalog: catalogs, schemas, permissions and lineage.
- Process files as they land with Spark Structured Streaming and Auto Loader.
- Work out why a Spark job got slow and fix it, using the Spark UI, partitioning, caching and broadcast joins.
- Ship your work with Git and a CI/CD pipeline instead of copying notebooks between workspaces.
- Model and build a Power BI report on your own gold layer, with DAX measures that give the right number.
Curriculum
22 modules, in the order they build on each other
Every module ends with something working in your repo. Open any one to see the full topic list and what you walk away with.
Phase 1 · Foundations
SQL coding practiceFrom SELECT to the queries interviews are actually built on.
- SELECT, WHERE and GROUP BY, and getting comfortable typing SQL instead of reading it
- Every join, including the ones people get wrong under interview pressure
- Subqueries, CTEs and window functions
- Deduplication, ranking, running totals and the top-N-per-group pattern
- Timed practice sets, because interview rounds are timed
Azure fundamentals and getting startedEnough Azure to work in it, without a certification detour.
- Subscriptions, resource groups, regions and how the billing actually works
- Azure Data Lake Storage Gen2, and the container and folder layout a lakehouse needs
- Access: service principals, managed identities and access keys
- Getting around the portal, and using the free credit so you are not paying to learn
- Where Databricks fits into the wider Azure picture
Phase 2 · Databricks and Spark
Databricks setupYour own workspace, wired to your own storage.
- Creating the workspace and connecting it to your storage account
- The workspace UI: notebooks, repos, catalog, jobs
- Notebooks, magic commands, and running Python and SQL in the same file
- Accessing data lake paths, and what belongs in DBFS versus a Unity Catalog volume
- Utilities, widgets and notebook parameters
Databricks computeClusters, and the settings that decide your bill.
- All-purpose clusters versus job clusters, and why the difference is money
- Node types, autoscaling, spot instances and auto-termination
- Serverless compute and SQL warehouses
- Cluster policies, libraries and init scripts
- Working out what a cluster costs before you start it, not after
Spark SQL fundamentalsHow the engine underneath all of this actually works.
- How Spark splits work across a cluster, in plain language
- Running SQL directly over files and over tables
- Lazy evaluation, transformations and actions
- The Catalyst optimiser, and why your query is not run the way you wrote it
Reading and writing data in SparkThe biggest module in the course, because this is most of the job.
- CSV, JSON, Parquet and Delta, and when each one is the right answer
- Schema inference versus an explicit schema, and why inference bites you in production
- Corrupt records and bad rows: permissive, dropMalformed and failFast
- Partitioned reads and writes, and the small-files problem
- Write modes: append, overwrite, error and ignore, and what each does to existing data
- Multi-line JSON, nested structs and arrays, and flattening them into something usable
Spark SQL tables and viewsWhere your data lives once it stops being a file path.
- Managed versus external tables, and what DROP TABLE deletes in each case
- Databases and schemas, and organising them so a team can find things
- Temporary views, global temp views and permanent views
- Creating tables from a query and from files
DataFrames and transformations, part 1The everyday API you will type more than any other.
- select, filter, withColumn, drop and alias
- Column expressions, and the difference between the SQL style and the DataFrame style
- Data types, casting, and handling nulls before they poison an aggregate
- Sorting, limiting and dropping duplicates
DataFrame transformations, part 2Joins, aggregations and the operations that cost you real time.
- Inner, left, right, full, semi and anti joins, and the broadcast join
- groupBy and aggregations
- Window functions in the DataFrame API
- union, pivot and explode
- Narrow versus wide transformations, and why a shuffle is the expensive one
FunctionsBuilt-in functions first, your own only when you must.
- The built-in Spark SQL functions worth committing to memory
- String, date and timestamp handling
- User-defined functions, and the performance reason to avoid them where you can
- Reusable Python functions, and keeping a notebook from turning into one long script
Phase 3 · Building the lakehouse
Delta Lake essentialsWhat turns a folder of Parquet files into something you can trust.
- ACID transactions on a data lake, and the exact problem they solve
- The transaction log, and time travel back to yesterday’s version
- MERGE for upserts, plus UPDATE and DELETE
- OPTIMIZE, Z-ORDER and VACUUM
- Schema enforcement and schema evolution
Medallion architectureBronze, silver, gold, and what genuinely belongs in each.
- The three layers, and the rule that keeps raw data raw so you can always reprocess
- Where cleaning, deduplication and business logic actually go
- Naming, folder layout and table conventions a second engineer can follow
- Building all three layers on your own dataset
Slowly changing dimensionsKeeping history instead of overwriting it.
- Type 1 and type 2, and the business questions each one can and cannot answer
- Surrogate keys, effective dates and current-row flags
- Implementing type 2 with a Delta MERGE
- Late-arriving and out-of-order records, which is where most implementations break
Unity Catalog essentialsGovernance, permissions and lineage across the whole workspace.
- Metastore, catalogs, schemas, tables and volumes
- Grants and access control, and sharing across workspaces
- Managed versus external locations, and storage credentials
- Lineage, audit logs and letting people discover data without asking you
- Migrating off the Hive metastore
Phase 4 · Pipelines and orchestration
Azure Data FactoryGetting data into the lake from systems that are not Databricks.
- Linked services, datasets, pipelines and integration runtimes
- Copy activity from SQL Server, REST APIs, blob storage and on-premise sources
- Parameters, variables and dynamic content, instead of twenty copy-pasted pipelines
- Triggers: schedule, tumbling window and event-based
- Calling a Databricks notebook from ADF and passing parameters into it
- Failure handling, retries, and monitoring runs when something breaks at 2am
Lakeflow JobsTurning notebooks into something that runs without you.
- Scheduling notebooks as jobs, and job clusters versus all-purpose clusters
- Multi-task jobs, dependencies, and passing values between tasks
- Retries, timeouts and alerting on failure
- Parameterised runs and repairing a failed run instead of rerunning everything
Spark Structured StreamingHandling data that keeps arriving.
- Streaming as an unbounded table, which is the idea that makes the rest easy
- readStream, writeStream, checkpoints and output modes
- Auto Loader, for files landing in the lake continuously
- Windowing, watermarks and late-arriving data
- When batch is honestly the better answer, and how to tell
Phase 5 · Engineering practice
Performance tuningThe job used to take four minutes. Now it takes forty. Find out why.
- Reading the Spark UI to see where the time actually went
- Shuffles, data skew, and salting a skewed key
- Partitioning, repartition versus coalesce, and getting file sizes right
- Caching, persisting and broadcast joins
- Adaptive Query Execution, and what it quietly fixes for you
Managing code with GitNotebooks are code. Treat them like it.
- Git basics: init, commit, branch, merge, push
- Databricks Repos, and connecting your workspace to GitHub
- Branching for dev, test and production work
- Merge conflicts in notebooks, which are considerably worse than in .py files
- Pull requests and code review
Automation and CI/CDShipping a change without anyone clicking Run in production.
- Why manual deployment breaks eventually, and usually on a Friday
- Separate dev, test and production workspaces
- Deploying notebooks and jobs automatically from the repo
- Automated tests on data pipelines, and what is worth testing
- Secrets, environment configuration and promoting a change through environments
Phase 6 · Reporting and capstone
Power BIThe part the business sees, built on the gold layer you made.
- Connecting Power BI to Databricks SQL and to your gold tables
- Star schema modelling, and relationships that give the right totals instead of plausible ones
- DAX: measures, calculated columns, CALCULATE and time intelligence
- Building a report someone would actually open twice
- Publishing, refresh schedules and row-level security
CapstoneOne pipeline, raw file to report, that you present and defend.
- Ingest with Data Factory into bronze
- Clean and model through silver and gold in Spark and Delta
- Type 2 history on a dimension, governed in Unity Catalog
- Schedule it, break it deliberately, and make it recover
- Build the Power BI report on top, then present the whole chain and answer questions on it
Portfolio
What you’ll have built
These are not practice exercises. They go in your repo, and you talk an interviewer through them.
End-to-end retail lakehouse
Sales, product and customer files land in bronze through Data Factory, get cleaned in silver, and become a gold star schema with type 2 history on the customer dimension.
Incremental loads with Delta MERGE
A daily load you can run five times in a row without producing a single duplicate row, using MERGE, watermarks and checkpoints.
Streaming ingestion with Auto Loader
Files landing continuously in the data lake, picked up by Structured Streaming and aggregated with windows and watermarks that handle late arrivals.
A governed, scheduled production pipeline
The whole chain running on a Lakeflow Jobs schedule, permissions set in Unity Catalog, deployed from GitHub through CI/CD rather than by hand.
The job that used to be slow
Take a deliberately badly written Spark job, read the Spark UI, find the skew and the shuffle, cut the runtime, and be able to explain exactly what you changed.
The Power BI report on top
A report on your gold layer with a proper star schema and DAX measures, published with a refresh schedule and row-level security.
Toolset
The stack you’ll be fluent in
Languages
Azure
Databricks
Lakehouse
Engineering practice
Reporting
Careers
Where this takes you
Azure Data EngineerData EngineerDatabricks EngineerETL Developer
Every data-driven company needs data engineers first. Azure Data Engineer is one of the most commonly advertised data titles in India, and the Spark and Delta skills carry over to any cloud.
Who it’s for
- Beginners with no data background. SQL starts at SELECT and Azure starts at creating the account.
- Testers, support engineers and developers who want out of their current role and into data.
- Analysts strong in Excel or SQL who want to build the pipelines instead of waiting on them.
- Graduates who need something concrete on GitHub before interview season.
What you need first
No prior data experience. The first module teaches SQL from the beginning and the second sets Azure up from a fresh account. Basic Python helps once you reach the Spark modules and is taught as you need it, so you do not have to go and learn Python separately first. You need a laptop, a steady internet connection, and an Azure account, which starts on free credit.
How we teach it
More than lectures — a route into the job
- Live real-world projectsYou learn by building deployable products, not slideware.
- 1:1 mentor guidanceIndustry mentors review your work and unblock you as you go.
- Placement supportResume, mock interviews and referrals until you land the role.
- Industry certificateA shareable certificate of completion plus a real portfolio.
FAQ
Straight answers
I have no IT background. Is this realistic for me?
Yes, if you put the hours in. Module 1 starts SQL at SELECT and Module 2 starts Azure at creating an account, so nothing is assumed. What the course does assume is practice: the people who get placed out of a data engineering track are the ones who rewrite the query until it runs, not the ones who watched a mentor do it. Tell the counsellor your background on the form and they will be straight with you about the timeline.
What does the course cost, and is there an instalment option?
The fee moves with the batch and with the mode you pick, so we do not print a number here that will be wrong next month. Send the form on this page and a counsellor gives you the current fee, the instalment options and the next batch dates, usually within a day. Talking to them costs nothing and commits you to nothing.
Will Azure and Databricks cost me money while I learn?
Very little. A new Azure account comes with free credit and Databricks has a trial on top of that. Cluster sizing, auto-termination and spot instances are taught early precisely so you do not leave a cluster running overnight and get a shock. If your credit does run out mid-course, your mentor will show you how to keep going on the cheap.
Why Azure and Databricks specifically, and not AWS?
Because Azure Data Engineer is one of the most commonly advertised data titles in India, so it is where the interviews are. And because the skills transfer: Spark, Delta Lake, the medallion pattern and dimensional modelling work the same way on AWS and GCP. Once you know this stack, moving cloud is a matter of learning different names for storage and access control, not relearning the job.
How is this different from the Cloud Data Engineering track?
This one goes deep on a single stack: Azure, Databricks, Delta Lake, Data Factory and Power BI, end to end, on one lakehouse you keep extending across 22 modules. Cloud Data Engineering is broader and spreads across AWS, Azure and GCP with Snowflake and Terraform in the mix. If you want "Azure Data Engineer" on your CV, start here.
Do I need to know Python before I start?
No. The Spark modules use PySpark and the Python you need is taught alongside it, at the point you need it. If you already write Python you will move faster through the DataFrame modules; if you do not, you will pick up enough to do the job, because data engineering Python is a narrow slice of the language rather than all of it.
How long does it take, and can I do it while working?
Twenty-two modules, with the Spark and lakehouse sections taking the longest because that is where the practice lives. Most people working full-time give it 8 to 10 hours a week. Weekday and weekend cohorts both run and sessions are recorded, so a missed class is not a lost one. Ask the counsellor for the current calendar before you commit to a batch.
Is it online or at the centre?
Both. There are in-person batches at the Sholinganallur centre in Chennai, and the same programme runs live online for learners elsewhere in India, with blended options in between. Choose the mode on the form and the counsellor sends the batch dates for that one.
What will I have to show at the end?
A GitHub repository with a working lakehouse in it: Data Factory pipelines, bronze, silver and gold notebooks, Delta tables with type 2 history, a Unity Catalog setup, a scheduled Lakeflow job, a CI/CD workflow and a Power BI report sitting on top. Plus a capstone you present and defend. That repository is what carries you through a technical screen.
Do I get placement support?
Yes, and it runs until you are hired: resume work against real data engineering job descriptions, mock technical interviews on SQL and Spark, and referrals. Across our tracks, 95% of learners have been placed. What we will not do is guarantee a named company or a number on your offer letter, because nobody can honestly guarantee that.
Keep exploring
More career-focused tracks
👋 Let's talk
Curious about enrollment procedures?
Get clear guidance on program selection, batch timing, fees, project exposure and career support.
