I Transitioned from Data Science to AI Engineering: Here’s Everything You Need to Know

isn’t dying, but it is evolving. Fast.

AI-related jobs are projected to grow ~40% year-over-year, creating over a million new roles by 2027.

In this article, I’ll take you through my transition from Data Science to Ai Engineering, as well as give you some practical advice on how to transition or to learn more about this area.

My path through Data Science to AI Engineering has been interesting and full of learnings. Here is a short snapshot of my journey so far:

I graduated from Physics and Astrophysics (bachelor’s and master’s) and transitioned to Data Science;
Performed two internships abroad in Data Science and Machine Learning;
Got my first full-time job as a Data Scientist in the biggest energy company of my country;
Transitioned to AI Engineering less than a year ago (as of May 2025) and now I work for a big logistics company.

If you are a data scientist, how often do you think about how your code reaches production? If the answer is ‘almost never’, AI Engineering might shock you.

Curious about how real-world experience in data science could shape your journey into AI engineering, or what surprising challenges I faced?

How is a daily life of an AI Engineer compare to a Data Scientist’s one?

What tools and platforms I use now, compared to before?

Keep reading it to know all about it!

Hello there!

My name is Sara Nóbrega, and I am a an AI Engineer.

I write about data science, Artificial Intelligence, and data science career advice. Make sure to follow me to receive updates when the next article is published!

Differences and Similarities Between Data Science and AI Engineering

AI Engineering is a very broad term and it may even include many Data Science tasks. In fact, it is often used as an umbrella term.

As a Data Scientist, I once spent 3 weeks tuning a model offline. Now, as an AI Engineer, we have 3 days to deploy it into production. Priorities shifted fast!

But does that mean that both roles are completely different and never overlap?

What if one day you want to apply to an AI Engineer role? Are data science skills transferable to the world of AI Engineering?

First, I will show you some findings of the research I did on this and then my personal intake and experience on the subject.

I did a little research for you…

From my investigation, the responsibilities of each role have broadened and converged over the past three years.

Data Scientist job descriptions today include more and more tasks besides analysis and model tuning. They often include: deploying models, building data pipelines, and applying Machine Learning Operations (MLOps) best practices.

Guess what, this is what I mainly do as an AI Engineer! (More on this in the next sections).

For example, a recent Data Scientist posting I saw explicitly required “experience with enterprise DataOps, DevSecOps, and MLOps”.

Until some years ago, data scientists focused mainly on research and modeling. Now, companies often expect data scientists to be “full stack”, which overwhelmingly means, fluent in almost everything.

This means that it is expected that data scientists have some knowledge of cloud platforms, software engineering, and even DevOps, so their models can directly support products.

One survey found 69% of data scientist job listings request machine learning skills and about 19% mention NLP, up from just 5% a year prior.

Cloud computing skills (AWS, Azure) and deep learning frameworks (TensorFlow/PyTorch) now appear in ~10–15% of data scientist ads as well, indicating a rising overlap with AI engineering skill sets.

There is a clear convergence in the skill sets of Data Scientists and AI Engineers. Both roles heavily use programming (especially Python) and data skills (SQL), and both need understanding of machine learning algorithms.

According to an analysis of 2024 job postings, Python is required in ~56–57% of both data scientist and ML engineer listings.

Cloud and MLOps skills seem to be the new common ground, as AI Engineers are expected to deploy on AWS/Azure and likewise “cloud skills will be essential” for future data scientists.

The table below highlights some core skills and how frequently they appear in job ads for each role, according to the sources that I list in the references section:

At first glance, the divergence is obvious. Data Scientist roles remain grounded in traditional data tasks: Python, SQL, general machine learning, and deriving insights from structured data.

ML/AI Engineers are positioned much closer to the world of software engineering. These professionals are tasked with taking experimental models and making them robust, scalable, and continuously deliverable.

But there is a clear convergence that is interesting and strategic.

We can see that cloud platforms are increasingly mentioned for Data Scientists, and MLOps tools are no longer confined to engineering roles. The skill sets are blending!

We’re seeing a trend where Data Scientists are being nudged closer to the engineering stack.

My Personal Journey and Intake

What did I do as a Data Scientist? What do I do know as an AI Engineering?

To give you some context, I worked as a data scientist in a big energy company, where my responsibilities revolved around developing time-series forecasting models (using XGBoost, LightGBM, SARIMAX, and RNNs), generating and validating synthetic data (via TimeGAN, statistical distributions, and imputation techniques), doing deep and extensive statistical analyses and employing machine learning models to tackle missing data in big data.

If you are interested, I wrote a ton of useful articles to deal with time-series data.

Some of the tools and platforms I used as a Data Scientist included: VSCode, Jupyter, MLflow, Flask, FastAPI, and Python libraries such as TensorFlow, scikit-learn, pandas, NumPy, Matplotlib, Seaborn, ydata-synthetic, statsmodels, and others.

In my previous internship, I would use PyTorch, Transformers, Weights & Biases, Git, and Python libraries for data distillation, supervised learning, applied statistics, computer vision, NLP, object detection, data augmentation, and deep learning.

The tools and platforms I use now

Python is still the main language I use. I do use Jupyter notebooks for prototyping, but most of my time is now spent writing Python code in VSCode (scripts, APIs, tests, etc).

My work is very connected to Microsoft Azure, particularly Azure Machine Learning, as my team uses it to manage, train, deploy, and monitor our ML models.

The entire MLOps lifecycle (from development all the way to deployment) runs in Azure. We also utilize MLflow to track experiments, compare different models and parameters and register all the model versions.

A major shift for me from DS to AI Engineering has been the consistent use of CI/CD tools, especially GitHub Actions. This was actually one of my first tasks when I started this job!

GitHub Actions allow me to build automated workflows that test and deploy ML models, so that they can be integrated into other pipelines.

Beyond machine learning, I also build and deploy backend components. For that, I work with REST APIs, with FastAPI and Azure Functions, to serve model predictions and connect them to our frontend applications or external services.

I have started working with Snowflake to explore and transform structured datasets using SQL.

Regarding infrastructure as a code, I have used Terraform to manage cloud resources as code.

Other tools I use include Git, Bash, and Linux environment. These are important for collaboration, scripting automation, troubleshooting, and managing deployments.

Some tasks I have performed as an AI Engineer

Now, I work as an AI Engineer for a big logistics company.

The first task I was assigned to was to improve and optimize continuous integration/continuous deployment (CI/CD) pipelines of ML models using GitHub Actions and Azure Machine Learning.

What does this mean in practice, you ask?

Well, my company wanted a reusable MLOps template that new projects could adopt without starting from scratch. This template is like a starter pack. It’s in a GitHub repo and has everything you’d need to go from a prototype in a notebook to something that can actually run in production.

Inside this repo, there is a Makefile (a script that lets you run setup tasks like installing packages or running tests with a single command), a CI workflow written in YAML (a file that defines exactly what happens every time someone pushes new code, for example, tests are run, and models get evaluated), and unit tests for both the Python scripts and the configuration files (to make sure everything behaves as expected and nothing breaks without us noticing).

If you wish to learn more about this, I actually wrote a full Dev Checklist for ML projects that describe these best practices, and that is totally beginner-friendly.

From linting and Makefiles to GitHub Actions and branch protection, it’s packed with the practical steps that I wish I knew earlier:

👉 Read it here: From Notebook to Production — A Dev Checklist for ML Projects

Unit tests are actually a core part of AI Engineering. They are often not the favorite task of anyone… but they are crucial for making sure things don’t break when your model hits the real world.

Because imagine you’ve spent days training a model, only to have a tiny bug in your preprocessing script mess everything up in production. Unit tests help catch those silent failures early!

But does this mean I have stopped performing Data Science tasks? Not at all!

In fact, one of my current tasks involves mapping departure and arrival times, cleaning route data, and integrating the results into a frontend app.

I think it is a perfect example of how Data Science (EDA, mapping, cleaning) blends with AI Engineering (integration, deployment awareness).

I want to highlight that both roles (Data Scientist and AI Engineer) can be quite broad and their responsibilities often vary from company to company, even sector to sector. What I’m sharing here is only based on my personal experience, which may not reflect everyone’s journey or expectations in these roles!

Collaboration Patterns

One thing I have noticed is that this overlap in responsibilities has forced closer collaboration with other team members. I have noticed that data scientists are increasingly working side-by-side with DevOps and backend engineers to ensure models actually run in production.

A study found that 87% of machine learning solutions fail to make it out of the lab without teams coordinating in an efficient way.

Over the last years, companies have recognized the need for collaboration. In fact, the need for MLOps best practices have come to life to bridge this gap between data scientists and DevOps.

Biggest Challenges So Far

I’m not gonna lie, this journey has been challenging. Everyone must be aware of the imposter’s syndrome, and I have certainly suffered from it as well. I guess it disappears over time as I feel I add value to the projects I participate in.

Right when I started to work as an AI Engineer, the biggest challenge was to get used to new tools, and to use them all together. As I was assigned an important task that only I was working on (the MLOps template one), I felt I had suddenly a lot of responsibility. I had to quickly learn the YAML language, Github Actions and how they connect to Azure.

Since I was really into MLOps, I ended up taking on the role of system architect in a few projects. I was responsible for figuring out how all the pieces would fit and work together, and then explaining it clearly to my managers.

I was definitely not used to these responsibilities and roles, but over time I’ve grown more confident in handling them.

Tips to transition from DS to AI Engineering

I would say that the first step to become an AI Engineer is to start by being interested and curious about how the big picture of AI works. This is how I started.

This is how I started!

I started by asking myself: How will this model actually go live to the users? How will it add value? How does the databases work, and how can we fetch the data in production? How can I make sure that in 6 months this model still works? How can I make sure that my model will be as accurate locally as in production?

Then, I started reading articles online and LinkedIn posts as well, before I transitioned to AI Engineering.

There is a huge amount of useful content online, for free. I also started taking some online courses so my skills become more solid.

If you are in a data science role, you could ask your manager to start contributing to production code in your team, or to include you in the meetings with the AI Engineers. From my experience, managers always like employees that want to learn more.

Then, you can learn online about GitHub Actions, Docker, and Azure/AWS. Learn about important production metrics like latency, uptime, monitoring

This is a very short roadmap, I will leave the rest of the tips for the next article 😉.

Final Word

My Mindset Shifted: Why AI Engineers Must Think Like Devs

To transition to an AI Engineering role, it is important to think about the big picture of a ML lifecycle: that is, to make sure the model will actually work, create impact and add value to the company.

What does this mean?

It means taking into account, during the whole lifecycle, how the model will be integrated into real-world systems — how it will be deployed, monitored, scaled, and maintained over time.

It means thinking beyond notebooks and training accuracy, and asking questions like: Where will this model run? How do we update it safely? What happens if the input data shifts next month?

For those entering or transitioning within the AI space, remember: you don’t need to master everything, but you do need to understand how your work fits into the larger picture of the ML lifecycle.

The deeper your empathy for the “other side” of the pipeline, the more impact you’ll have.

As you noticed through this article, transitioning to AI Engineering for me has been about working on, learning about and owning the entire ML lifecycle, not just the model training.

In my past role as a data scientist, I was performing traditional DS tasks like EDA, anomaly detection, data wrangling, model development and packaging. Indeed, it was directly connected to what I had learned in university.

As an AI Engineer, I feel my daily tasks are a blend of both roles. I still explore and clean data, but I feel I need to think like a dev, so I am sure the models work in production and maintained over time.

Definitely one of the biggest mindset shifts was learning how to ship code ready for production and also to develop a mindset of automation: automate installations, testing, deployment, monitoring.

It has been an interesting journey so far, that I intend to document and share further on.

Thank you for reading! Hope you found this post useful.

🔔 One more thing!

I also write a free newsletter, Sara’s AI Automation Digest, where I share monthly insights, tools, and behind-the-scenes takes on AI, automation, and how it’s transforming the way we work.

Subscribe now and get access to my FREE AI Tools Library — a curated Notion database of 20+ AI tools with real-world use cases, features, and limitations.

I offer mentorship on career growth and transition here.

If you want to support my work, you can buy me my favorite coffee: a cappuccino. 😊