What Happened to the DBA?

Jesse Anderson from Big Data Institute and David Moise Discuss:
What happened to the DBA?
Data Teams
How the Big Data Institute Finds Data Scientist and Data Engineers for their team

— Contents of this Video—

00:00 – Intro
1:06 – What happened to the DBA?
1:47 – DBA – New job for the 21st Century?
2:29 – Why we needed so many DBAs
3:10 – What we do instead of DBAs
4:10 – Software Engineers & Databases
5:10 – Data Jobs for the next 10 Years
6:22 – The Job Market for Data Engineers
7:44 – Where and How do you find Data Engineers
8:56 – Building Data Teams – Data Science, Data Engineers & Operations
10:30 – What HR needs to know about Data Science
— End of Contents—

Jesse Anderson – https://www.linkedin.com/in/jessetanderson/
Bis Data Institute – https://www.bigdatainstitute.io/

DATA TEAMS / A Unified Management Model for Successful – Data-Focused Teams – https://www.datateams.io

David Moise – https://www.linkedin.com/in/dmoise/
Technical Talent Strategies – https://decideconsulting.com/tech-talent-strategy/

This morning, I am speaking with Jesse Anderson,
and Jesse is going to introduce himself and
tell us a little bit about him.
And we’re going to talk about some interesting
things when it comes to hiring technology people
and specific, specifically about DBAs and data science.
So, Jesse, take it away. Just who are you? Your company.
What do you do?
Thank you so much for having me.
My name is Jesse Anderson.
I’m managing director at Big Data Institute.
We mentor companies all over the world on big
data, but not just from a technical point of
view, but also from the organizational point of view
that we’re about to talk about.
What should the team be like to that end?
I’ve done pretty extensive research on the subject.
I’ve written three books.
My latest book is called Data Teams, and I highly
suggest everybody read it because I talk about some of
the things in more depth we’ll be talking about today.
Okay. Great. I appreciate that.
We’ll talk about some of those books in a bit here.
All right.
So one of the topics that got us together
in the first part or in the first place,
was what has happened to the DBA database administrator
used to be a very hot role.
Lots of companies hiring for them.
We just don’t see that as much anymore.
You still see it out there under jobs, but
it’s a small percentage of jobs compared to how
it used to be, what’s going on with that?
And what two cent do you have to
provide about what’s happened to the DBA?
Where is that role going to now?
So it’s interesting that that was happening because when I
looked at that and you and I were probably on
the same sort of emails and news programs or news
articles, they were saying, DBA the new job for the
20th century or the 21st century.
And I was thinking, no, that is not the one.
And in fact, I spoke at a College class of people who
are going to graduate as DBAs and said, Is it too late?
Could you change, like, being really the
bearer of bad news money on tuition?
It was kind of sad to have to stand up
there and say that because it hadn’t happened yet.
There wasn’t this thing that somebody could point to
and say, oh, yes, this is what’s happening.
And so it was a different sort of thing that it
was me saying, no, I see what’s happening in the field.
So here’s kind of let me break down.
What’s happening or what happened is part of the
reason we needed so many DBAs is we were
beating the hell out of our databases, and we
are beating our databases down because we didn’t have
other systems to do the analytics we needed.
So we had the database and we would use
it for everything instead of saying, Well, the database
does this thing really well but we need this
other technology, whether that’s spark, whether that’s Hadoop or
whatever to do other parts.
And so once we took the load off our
production databases and put them onto these other systems,
we didn’t need a 1020 person DBA team.
We had other technologies we could use.
So this was a key change that I
don’t think most people saw at that time.
Okay, that’s interesting.
So what tools are being used right now is it just
being shot off to the data engineers, the data scientists?
Where is that being done?
A lot of times the actual engineering work
is being offloaded onto the data engineering team.
So where it was data was that simple or everything?
Data was in the database engineers, the
database administrators, we’ve now shifted that some
data is the responsibility of our data
engineers and data engineering team.
A lot of the actual responsibility for the
infrastructure falls on the data engineering team.
But now that responsibility for the actual
analytics that still falls to our business
intelligence team, our data scientists.
But now they have better tools to be able to decide.
I want to run the analytics here versus over here.
Okay, how do you feel about, like, software
developers get sticking their hands in the database?
Good thing. Bad thing.
Are they better at it than they used to be in trouble?
Oh, no.
I’ve actually talked about this because software engineers,
if you’re having to run a production database,
you’re going to have all sorts of problems.
This is not who they are.
Conversely, if you’re going to have DBAs magically be
changed into data engineers, so just for clarity, my
definition of a data engineer is a software engineer
who has specialized their skills in big data.
So from that definition, if you magically take your
DBAs or even data warehouse engineers and say, I
dub the data engineer, it doesn’t magically change their
skill set, doesn’t change their abilities.
You’ve just created a different problem.
You shifted the problem.
So to answer your question, it is
very imperative that your data engineers be
software engineers, in my opinion.
Very good answer.
So if you were to go into one of
those College classrooms today where they’re teaching people something,
what would you say to them about what jobs
to focus on for the next ten years?
I think the job to focus on the next ten years
is enabling data and to enable data the right way.
Where if one of the issues we have
with sort of DBA was a significant limitation
of skills, skill was only around SQL.
If it could be expressed in SQL, they could do it.
However, for the more advanced types of data,
we’re going to need people who can program.
So it’s people who can program who can also do SQL.
But even more importantly, understand business.
They understand the business problem.
So if we have somebody who’s been understanding all three
parts of that, maybe they’re a data engineer, and they
understand that in a cursory way, still so much better
off somebody who only understands the code.
So I think it’s really key is saying, yes,
we will specialize, but cross train as much as
you can, you’ll be so much better off. Okay. Understood.
So one of the things I always like
to talk about is just the current market.
You’re hiring a lot of the data
oriented people, data scientists, data engineers.
How’s the market change for finding people these
days compared to pre pandemic levels, pre pandemic?
It was difficult to hire people because there
weren’t very many post pandemic current pandemic.
It’s even more difficult to hire people.
There’s even higher demand.
And what’s happened is the companies that
are hiring for engineers for data scientists,
they’re seeing even more importance of data.
So the pandemic has put even more emphasis on data.
And because of that, they’re hiring
even more people of data.
And there weren’t enough people to go around before.
And now there’s even fewer people to go around.
That said, we still have a glut of
people who don’t know what they’re talking about.
Quite frankly, we still have a lot of
data scientists, a lot of data engineers who
just changed a title and haven’t learned.
So my advice to them is buckle down, learn your
skills, understand this stuff, and you can’t get a job.
But don’t just try to lie your way
or fudge your way through an interview.
It’s not good for either side. Yeah. Understood.
Have you done anything?
What tips do you use or what things do you do
to try and find some people to join your company?
Have you found anything that works for you that
any secrets that other companies could take and use
to try and find some technical people?
It’s using your network.
Do you have a good reputation?
Are you out there talking about this?
In our case, I’ve spoken pretty extensively about this.
I know a lot of people.
Some of my former students, I know them.
And I saw what they did during that class, and I’m
able to say, oh, I would actually hire that person again.
So I saw how they learned.
I saw how hard they worked.
So I think that was key.
So having a good presence out there in the community,
having a good name out there in the community.
One thing I have put forward to companies and said, you
can hire people with the makings of some of these things,
but you will need to give them the tools.
And so you may want to become known as the
company that will hire and actually give the resources for
them to become full fledged data engineers, for example.
Okay.
And you mentioned the book data teams.
Can you give us a premise of
the book and who should read that? Sure.
Most of the books and data
have all been management focused.
Excuse me, not management focused here.
Is about this technology.
And here’s how you run this technology.
And what I saw was this need for a book
that said, Here is how you run the teams.
And that was the book I wrote.
I wrote the book.
The book is called Data Teams,
a Unified model for managing teams.
And what I say there is most of the
time management has gotten this message loud and clear
that data science, data science, data science.
And so they think just hire a data
scientist and everything’s good in the book.
I try to say no, you actually
need three teams to do this, right.
You do need the data science.
Yes, but you also need data engineering and operations.
And I try to make that clear in
there of what is data science, what is
data engineering and what is operations.
And then I go a step further.
I didn’t want to book those theoretical, although
it comes from my experience consulting with companies.
I did several case studies of companies who actually do this
on a day to day basis, and I asked them over
a ten year period, what does this look like?
Is that body of work did not exist anywhere.
Okay.
Interesting stuff.
All the companies I’m speaking with, I think that’s some
good advice that a lot of people need to hear.
So we’ll try and get a link and plug that
there was one part that I forgot to say.
And I wrote part specifically for professionals like
yourselves, HR people, because there was a missing
part that spoke to HR and said, this
is what this person should be.
This is what this job description should be.
And this is how much you should be paying them.
Because this is all what was missing when we
work with companies, the HR Department would say, but
I should be paying this data engineer 40,000.
Let’s say us that’s never going to happen.
The HR people didn’t understand.
Oh, data engineer means this and this and this.
And it’s actually a
specialization of software engineering.
So I put that in print so
that a manager could say, Here HR.
Here’s what it says.
We have it in print.
Please listen. Important stuff.
I mean, there’s new roles, new jobs,
new functions coming online more and more.
How many times have we heard data is the new oil?
We’ve heard that a lot algorithms of the new
oil or the algorithms of the new data.
So the phrase I’ve heard before, but now
I appreciate your time and coming in.
Just so people have heard this
and are kind of interested.
Who should call you who’s your target about
who you’re trying to reach for your company?
We’re looking for two types of people, people
who are running a project right now, and
they’re stuck, and they don’t know why.
It’s exactly what we do.
We have the experience to come in figuring
out what’s going on and fix it.
And then other people who hear about these problems and
say, I don’t want that problem and they’re starting that
team, and they want to start it right?
I want to talk to those two people.
Sounds like a good idea.
Reaches at Big Datainstitute IO okay.
We’ll definitely have a link.
And everywhere that we share this. All right.
Jesse, thanks for your time. I appreciate.
Thank you so much