Job Description

Data Engineer

Requirements:

• Strong data engineering fundamentals.
• Utilize big data frameworks like Spark/Databricks.
• Training LLMs with structured and unstructured data sets.
• Understanding of Graph DB.
• Experience with Azure Blob Storage, Azure Data Lakes, Azure Databricks.
• Experience implementing Azure Machine Learning, Azure Computer Vision, Azure Video Indexer, Azure OpenAI models, Azure Media Services, Azure AI Search.
• Determine effective data partitioning criteria.
• Utilize data storage system spark to implement partition schemes.
• Understanding core machine learning concepts and algorithms.
• Familiarity with cloud computing skills.
• Strong programming skills in Python and experience with AI/ML frameworks.
• Proficiency in vector databases and embedding models for retrieval tasks.
• Expertise in integrating with AI agent frameworks.
• Experience with cloud AI services (Azure AI).
• Experience with GIS spatial data to create markers on maps (lat long nearest topology of road, geo-locate between datasets, correlation etc.).
• Experience with Department of Transportation Data Domains developing an AI Composite Agentic Solution designed to identify and analyze data models, connect & correlate information to validate hypotheses, forecast, predict and recommend potential strategies and conduct What-if analysis.
• Bachelor's or master's degree in computer science, AI, Data Science, or a related field.

Job Tags

Similar Jobs