Big Data Engineer
Sacramento, CA
3-6 Months
Rate: $50/hr on C2C
Interview Mode: Phone + F2F
Any visa fine
Description:
Create an optimal data pipeline architecture.
Assemble large, complex data sets that meet functional / non-functional business requirements.
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, designing infrastructure for greater scalability.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
Create detailed step-by-step documentation of data architecture from both the macro view (i.e. architectural diagrams) and the micro view (i.e. script level tasks in Airflow).
Qualifications:
- Advanced SQL knowledge and experience working with relational databases, query authoring and optimization (SQL) as well as working familiarity with a variety of databases.
- Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
- Build processes supporting data transformation, data structures, metadata, dependency and workload management.
- A successful history of manipulating, processing and extracting value from large structured and unstructured datasets.
- Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
- Experience with big data tools: Snowflake, Hadoop, Spark, Kafka, etc.
- Experience with relational SQL and NoSQL databases.
- Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
- Experience with AWS cloud services: EC2, VDI, and RDS.
- Experience with object-oriented/object function scripting languages: Python, R, Java, C++, Scala, etc.
Knowledge/Skills and Abilities:
- Self-directed and proactive problem-solver.
- Confident verbal communicator.
- Ability to effectively and clearly communicate complex information in writing.
- Strong project management and organizational skills.
- Experience supporting and working with cross-functional teams in a dynamic environment.
- Ability to establish and maintain external partnerships/relationships.
- Demonstrated ability to prioritize, multi-task, and effectively manage multiple tasks simultaneously.
- Exceptional organizational skills and attention to detail.
- A customer service orientation.
- Ability to easily transition to/from independent and collaborative tasks.
- Experience with Salesforce preferred.
- Experience working in K-12 data and education highly desirable.
- Familiarity with a variety of K-12 student information systems (SIS) like PowerSchool, Aeries, Illuminate, etc.… is a plus
- A commitment to increasing educational access and equity.
Education and Experience:
- We are looking for a candidate with 3+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.
Ashok
[CONTACT]
*** EXT 222
