Designing Cloud Native Apps in GCP

## Selecting the appropriate Cloud Service Model - Infrastructure-as-a-service (IaaS) model - large degree of fliexibility in implementation - requries a significant amount of labor - Software-as-a-service (SaaS) model - higher velocity for delivery of services while maintaining a fair amount of flexibility - at the expense of flexibility - Iaas -> CaaS -> PaaS -> Faas -> Saas (from low fliexibility to high velocity) ## Protability and Design Consideration - Protable Languages - Platform considerations - Platform specific designs are targeted for a specific environment - Vendor lock in ## Evaluating Different Service Technologies - Google App Engine - you want to focus on writing code, never wants to touch a server, cluster or infrastructure - you want to build a highly reliable and scalable serving app or component without doing it all yourself - you value developer velocity over infrastructure control - Minimize operational overhead - Google Kubernetes Engine - you want to increase velocity and improve operability dramatically by separating the app from the OS - you need a secure, scalable way to manage containers in production - you don't have dependencies on a specific operating system - Google Compute Engine - you need complete control over your infrastructure and direct access to high-performance hardware such as GPUs and local SSDs - you need to make OS-level changes, such as providing your own network or graphic drivers, to squeeze out the last drop of performance - You want to move your application from your own colo or datacenter to the cloud without rewriting it - You need to run a software package that can't easily be containerized or your want to use existing VM images ## Operating System Considerations - CentOS - Container-optimized OS from Google - CoreOS - Debian - Red Hat Enterprise Linux (RHEL) - SUSE Enterprise Linux - SLEX for SAP - Ubuntu - Windows Server ## Location of Your Service Components - to cut down on latency and provide better services to your end users ## Microservice Architectures - Separated into independent constituent parts, with each part having its own realm of responisbility - Refactored from monolithic apps that have very tight coupling to a micro-service based architectures - Advantages: - the code base becomes more modular and easier to manage - it becomes much easier to reuse services for other applications - it is much easier to scale and tune individules services ## Defining Key Structures - avoid monotonically increasing keys - instead, migrate to keys that use random numbers, such as UUID ## Session Management - keep a session cache - Cloud Spanner - a limit of 10K sessions per database per node - a client can delete a session - The Cloud Spanner database service can delete a session when the session is idle for more than 1 hour ## API Management Consideration - Apps should be designed to have loosely coupled components - Pub/Sub model enables event-driven architectures & asyn parallel processing - Pulisher---(publish event)---> Event Channel --(Fire Event)---> Subscriber - Pulisher---(publish event)---> Event Channel <--(Subscribe)---- Subscriber ## Health Checks - Cloud Load Balancing - Interal Load Balancing - TCP Proxy Load Balancing - SSL Proxy Load Balancing - HTTP(s) Load Balancing - Stackdriver Monitoring --- uptime check ---> storage/database/network - Example: ``` gcloud compute health-checks create [PROTOCOl] [HEALTH_CHECK_NAME] \ --description=[DESCRIPTION] \ --check-interval=[CHECK_INTERVAL] \ --timeout=[TIMEOUT] \ --healthy-threshold=[HEALTHY_THRESHOLD] \ --unhealthy-threshold=[UNHEALTHY_THRESHOLD] \ ...additional flags ```

Auto-upgrading nodes in GCP Containers

## Checking the state of auto-upgrade for an existing node pool ``` gcloud container node-pools describe node-pool-name \ --cluster cluster-name \ --zone compute-zone ``` ## Enabling node auto-upgrades for an existing node pool ``` gcloud container node-pools update node-pool-name --cluster cluster-name \ --zone compute-zone --enable-autoupgrade ``` ## Disabling node auto-upgrades for an existing node pool where: ``` gcloud container node-pools update node-pool-name --cluster cluster-name \ --zone compute-zone --no-enable-autoupgrade ``` - node-pool-name is the name of the node pool. - cluster-name is the name of the cluster that contains the node pool. - compute-zone is the zone for the cluster.

Azure Synapse Analytics 101

- formerly Azure SQL Data Warehouse (SQL DW) - Massively Parallel Processing (MPP) Data Warehouse - Connection Security - Firewall rules are used by both the server and the database to reject connection attempts from IP addresses that haven't been explicitly whitelisted. - Authentication - SQL pool currently supports SQL Server Authentication with a username and password, and with Azure Active Directory. - Authorization - Authorization privileges are determined by role memberships and permissions. Authorization privileges are determined by role memberships and permissions. - Data Encryption - protects against the threat of malicious activity by encrypting and decrypting your data at rest. Associated backups and transaction log files are encrypted without requiring any changes to your applications when encrypting your database. - Advanced Data Security - provides a set of advanced SQL security capabilities, including data discovery & classification, vulnerability assessment, and Advanced Threat Protection. ## Transparent Data Encryption (TDE) - It helps protect against the threat of malicious activity by encrypting and decrypting your data at rest. When you encrypt your database, associated backups and transaction log files are encrypted without requiring any changes to your applications. TDE encrypts the storage of an entire database by using a symmetric key called the database encryption key. ## Granular access controls - Granular Permissions let you control which operations you can do on individual columns, tables, views, schemas, procedures, and other objects in the database. Use granular permissions to have the most control and grant the minimum permissions necessary. - Database roles other than db_datareader and db_datawriter can be used to create more powerful application user accounts or less powerful management accounts. The built-in fixed database roles provide an easy way to grant permissions, but can result in granting more permissions than are necessary. - Stored procedures can be used to limit the actions that can be taken on the database. ## Service Type - Compute Optimized Gen1 - Compute Optimized Gen2 ## Scalability - Linear Scale on data warehouse unit ## Backup - Use data warehouse snapshot to create a restore point

Deleting objects in GCP using Lifecycle Policy File

Example: Delete the object after 31 days. ```json { "rule": [ { "action": {"type": "Delete"}, "condition": {"age": 31} } ] } ``` Set the policy ``` gsutil lifecycle set life.json gs://$BUCKET_NAME_1 ``` Get the policy ``` gsutil lifecycle get gs://$BUCKET_NAME_1 ```

GCP Storage 101

- Use cases: - website content - storing data for archiving and disaster recovery - distributing large data objects to users via direct download - Scalable to exabytes - Time to first byte in milliseconds - Very high availability across all storage classes - Single API across storage classes ## Signed URLs - Valet key access to buckets and objects via ticket: - crytographically signed URL - time-limited - operations specfied in ticket: HTTP, GET, PUT, DELETE (not POST) - any user with URL can invoke permitted operations - Example: - ``gsutil signurl -d 10m path/to/privatekey.p12 gs://bucket/object`` ## Strong Global Consistency - read-after-write - read-after-metadata-update - read-after-delete - bucket-listing - object-listing - granting access to resources ## Choose among Cloud Storage classes   ## Cloud Bigtable - Fully managed NoSQL, wide-column database service for terabyte applications - Accessed using HBased API - Native compatibility with big data Hadoop ecosystems - Managed, scalable storage - Data encryption in-flight and at rest - Control access with IAM - Bigtable drives major applications such as Google Analytics and Gmail ## Cloud SQL - managed RDBMS - offers MYSQL and PostgreSQLBeta databases as a service - automatic replication - managed backups - vertical scaling (read & write) - horizontal scaling (read) - google security - enable binary logging to use point-in-time recovery but it may slightly affect write performance ## Cloud Spanner - horizontally scalable RDBMS - strong global consistency - managed instances with high availablity - SQL queries - ANSI 2011 with extensions - automatic replication ## Cloud Datastore - designed for application backends - supports transactions - includes a free daily quota # Comparing Storage Options   # Decision Flowchart