Databricks Administrator/Architect

A

ACCEL BI CORPORATION

Databricks Administrator/Architect

Raleigh, NC
Full Time
Paid
  • Responsibilities

    Benefits:

    401(k) matching

    Dental insurance

    Health insurance

    Paid time off

    We are seeking a highly skilled Databricks Administrator/Architect to develop, optimize, and maintain the Databricks environment supporting the organization’s data initiatives. The candidate will design, plan, and keep the Databricks environment, working closely with DBA and development teams to ensure the platform aligns with business needs.

    This is a Hybrid role

    Key Responsibilities:

    Design, implement, and manage Databricks clusters, ensuring platform scalability, performance, and security.

    Provide escalated support for Databricks users and oversee role-based access control using Unity Catalog.

    Conduct regular cluster health checks, implement best practices, and maintain platform documentation.

    Design and implement tailored data solutions involving API data ingestion, data pipelines, analytics, and beyond within a continually evolving tech stack.

    Manage on-prem data ingestion into Azure using Azure Data Factory (ADF).

    Build and maintain data pipelines using medallion architecture to clean, transform, and aggregate data from various sources.

    Basic Qualifications:

    5+ years of extensive hands-on experience implementing Lakehouse architecture using Databricks Data Engineering platform, SQL Analytics, Delta Lake, and Unity Catalog

    5+ years of strong understanding of Relational and Dimensional modeling

    6+ years of proficiency in coding with Python, SQL, and PySpark to prioritize performance, security, scalability, and robust data integrations

    2+ years of experience implementing serverless real-time/near real-time architecture using Cloud (Azure, AWS, or GCP) and Spark technology (Streaming & ML)

    4+ years of experience with Azure infrastructure configuration, networking, building large data ingestion pipelines, and conducting data migrations using ADF or similar technology

    7+ years of experience working with SQL Server features, including SSIS and CDC

    2+ years of experience with Databricks platform, security features, Unity Catalog, and data access control mechanisms

    4+ years of experience with GIT code versioning software

    Preferred Qualifications:

    Databricks Certifications