Google Cloud Certified Professional Data Engineer

• Candidates should be well-versed in data management, processing, and security, as well as data warehousing. They should also have practical expertise with the Google Cloud Platform.
• Strong programming skills in SQL and at least one other programming language
• Knowing Hadoop, MapReduce, and Spark, as well as other related distributed systems technologies
• Working knowledge of big data analysis and processing tools including Apache Beam, Apache Flink, and Apache Spark
• Expertise with the Google Cloud AI Platform and understanding of machine learning methods
• Knowledge of cloud computing ideas and familiarity with cloud-based services and infrastructure

Building and operationalizing storage systems

• Effective use of managed services (Cloud Bigtable, Cloud Spanner, Cloud SQL, BigQuery, Cloud Storage, Datastore, Memorystore)
• Storage costs and performance
• Life cycle management of data

Building and operationalizing pipelines

• Data cleansing
• Batch and streaming
• Transformation
• Data acquisition and import
• Integrating with new data sources

Building and operationalizing processing infrastructure

• Provisioning resources
• Monitoring pipelines
• Adjusting pipelines
• Testing and quality control

Designing for security and compliance

• Identity and access management 
• Data security (encryption, key management)
• Ensuring privacy 
• Legal compliance 

Ensuring scalability and efficiency

• Building and running test suites
• Pipeline monitoring 
• Assessing, troubleshooting, and improving data representations and data processing infrastructure
• Resizing and autoscaling resources

Ensuring reliability and fidelity

• Performing data preparation and quality control 
• Verification and monitoring
• Planning, executing, and stress testing data recovery (fault tolerance, rerunning failed jobs, performing retrospective re-analysis)
• Choosing between ACID, idempotent, and eventually consistent requirements

Ensuring flexibility and portability

• Mapping to current and future business requirements
• Designing for data and application portability 
• Data staging, cataloging, and discovery

Selecting the appropriate storage technologies

• Mapping storage systems to business requirements
• Data modeling
• Trade-offs involving latency, throughput, transactions
• Distributed systems
• Schema design

Designing data pipelines

• Data publishing and visualization 
• Batch and streaming data 
• Online (interactive) vs. batch predictions
• Job automation and orchestration

Designing a data processing solution

• Choice of infrastructure
• System availability and fault tolerance
• Use of distributed systems
• Capacity planning
• Hybrid cloud and edge computing
• Architecture options 

Migrating data warehousing and data processing

• Awareness of current state and how to migrate a design to a future state
• Migrating from on-premises to cloud (Data Transfer Service, Transfer Appliance, Cloud Networking)
• Validating a migration

Leveraging pre-built ML models as a service

• ML APIs 
• Customizing ML APIs 
• Conversational experiences 

Deploying an ML pipeline

• Ingesting appropriate data
• Retraining of machine learning models (AI Platform Prediction and Training, BigQuery ML, Kubeflow, Spark ML)
• Continuous evaluation

Choosing the appropriate training and serving infrastructure

• Distributed vs. single machine
• Use of edge compute
• Hardware accelerators 

Measuring, monitoring, and troubleshooting machine learning models

• Machine learning terminology 
• Impact of dependencies of machine learning models
• Common sources of error