It’s an in-demand position, with fewer qualified people in the job market than in other sectors of tech. Job posting/reviewing site Glassdoor named it the number one job in the United States with a base salary starting at just over six figures.
Why is this job such a valuable position? This is due to the “big data” trend. It entails sifting through the mountains of information that we produce every moment of our online lives. For example, years ago, OkCupid, the online dating service, was sifting through data, and when they laid the number of users logged in over the weather patterns of a region, they noticed a huge jump in activity from users in a city every time that city got rain. Now, when they pull a local weather report into the system, they automatically send out an email to the users in the city to let them know this rainstorm is the best time to meet someone else who will be online the same time as you. That’s big data. It is as important for Wall Street as it is for the Centers for Disease Control as it is for a clothing store trying to get their advertising to the person that is most likely to want to buy from them.
But it takes a lot of digging through endless amounts of numbers and comparing it to an endless number of factors to make sense of the information. So it’s no surprise to anyone that there would be a plethora of educational institutions from online programs to Ivy League universities that are offering curriculum in data science. These can be very expensive programs.
Enter Databricks, a startup tech company that is currently one of the biggest names in “big data” science. They are soon to roll out a new program that allows students a free version of their data cloud service called the “Community Edition.” They are currently accepting applications for the wait list, where students and others will have access to a micro-cluster as well as a cluster manager and notebook environment. They will also have access to Spark training resources. Spark is the name of the Databricks technology they developed while the founders were professors at UC Berkeley.
A year ago Databricks unveiled a free online course to train people in a data science using Spark, and the demand went through the roof. 20,000 or so people have completed the course and over 76,000 people have enrolled in it. “We wanted to double down on this. We thought, how can we teach more people, teach 100,000 people how do data science? That’s what we’ve been working on for six months, this version of Databricks,” CEO Ali Ghodsi said.
If you are interested in teaching yourself Spark, in adding to your developer, engineer, or IT portfolio, I highly recommend you read more about Databricks and consider signing yourself up for the waitlist!