Job Description
Job Description Job Summary Responsible for collecting, storing, processing, and analyzing large sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. He/she will also be responsible to follow architecture and best practices used across the enterprise. Knowledge/Skills/Abilities: Define ideal Architecture, Evaluating tools and Frameworks, Standards & Best Practices for implementing scalable business solutions Implement Batch and Real-time data ingestion/extraction processes through ETL, Streaming, API, etc., between diverse source and target systems with structured and unstructured datasets Design and build data solutions with an emphasis on performance, scalability, and high-reliability Code, test, and document new or modified data systems to create robust and scalable applications for data analytics Build data model for analytics and application layers Working closely with multiple teams and Business partners, for collecting requirement and providing optimal solution Proven experience on Hadoop cluster components and services (like HDFS, YARN, ZOOKEEPER, AMBARI/CLOUDERA MANAGER, SENTRY/RANGER, KERBEROS, etc.) Ability to participate in troubleshooting technical issues while engaged with infrastructure and vendor support teams. Job Qualifications REQUIRED QUALIFICATIONS: 3 years of progressive experience using data, building ETLs (extract, transform, load), using disparate data sources for data models and analysis, and familiarity with data visualization tools and scripting tools such as Python, Ruby or Perl. Experience in building stream-processing systems, using solutions such as Kafka, Storm or Spark-Streaming Experience on Big Data tools such as, Spa