Big Data Developer

Position: Big Data Developer

Job Description: The Big Data Developer will be responsible for designing, implementing, and maintaining scalable and efficient big data solutions. They will work closely with data scientists, data analysts, and other stakeholders to ensure optimal data pipeline architectures and data set processes.

Key Responsibilities

• Design and develop big data solutions using Hadoop, Spark, or other big data technologies.
• Develop and maintain ETL processes to transform and load data from various sources into the data lake or warehouse.
• Implement data security and privacy measures to protect sensitive data.
• Optimize and tune the performance of big data applications and queries.
• Collaborate with data scientists and analysts to understand data requirements and implement data models.
• Monitor and troubleshoot data pipeline issues and ensure data integrity and availability.
• Document code, solutions, and processes to ensure maintainability and knowledge sharing.
• Stay updated with the latest big data technologies and trends to propose new solutions or improvements.
  
  

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or related field. Master’s degree preferred.
• Proven experience as a Big Data Developer or similar role.
• Strong proficiency in big data technologies such as Hadoop, Spark, Kafka, Hive, or similar.
• Experience with programming languages such as Java, Scala, Python, or others used in big data development.
• Knowledge of SQL and NoSQL databases.
• Experience with data warehousing solutions and cloud platforms like AWS, Azure, or Google Cloud.
• Strong analytical, problem-solving, and communication skills.
• Ability to work in a fast-paced environment and manage multiple tasks simultaneously.
• Familiarity with agile methodologies and DevOps practices is a plus.
  
  

Preferred Skills

• Certification in big data technologies or related fields.
• Experience with machine learning frameworks and tools.
• Knowledge of data visualization tools such as Tableau or Power BI.
• Experience with containerization and orchestration tools like Docker and Kubernetes.
  
  

Personal Attributes

• Team player with excellent interpersonal skills.
• Self-motivated with a strong commitment to continuous learning.
• Detail-oriented with a focus on delivering high-quality solutions.
• Ability to adapt to changing priorities and requirements.