devkuma – Cloud

What is Cloud?

kc@example.com (kc kim) — Fri, 30 Dec 2022 21:35:19 +0900

What is cloud?

Recently, the term cloud, or cloud computing, has become common. What does cloud mean? Cloud means a cloud in the sky.

Until now, photos taken with digital cameras or smartphones, documents created on PCs, music, videos, and similar data were naturally stored on a user’s own PC or smartphone.

Data saved on a user’s own PC or smartphone could only be viewed from that device. Sharing between PCs is possible if sharing is configured, but basically the data cannot be viewed elsewhere.

For example, to view photos taken with a smartphone on a PC, they had to be transferred to the PC.

Data cannot be viewed on another computer.

Cloud appeared to solve this. With the emergence of cloud, data that had been stored on PCs or smartphones can now be stored in the cloud.

Saving data in the cloud makes it viewable from other computers.

By saving data in the cloud, it became possible to view it from many devices. Photos taken with a smartphone can be viewed easily after uploading them to the cloud, without transferring them to a computer.

From a user’s perspective, some may wonder where data stored in the cloud is actually located. Because it is like an invisible cloud, it is called cloud.

In addition to storing data, cloud services provide applications, environments for running applications, and servers or infrastructure for running applications. Details are introduced in the cloud service section below.

What is a cloud service?

Cloud services provide services to users through a network by using the cloud. Cloud services include the following types.

Source: https://www.inap.com/blog/iaas-paas-saas-differences/

SaaS (Software as a Service)

SaaS is a service that provides applications through a network.

Originally, to run an application on a PC, the application had to be installed on the PC.

SaaS (Software as a Service)

With SaaS, users can connect to cloud applications through the internet and use them without installing applications on their computers.

Representative SaaS examples include Gmail and Microsoft Office 365.

PaaS (Platform as a Service)

PaaS provides a development environment, or platform, for running applications and programs.

In the past, a server was required to run an application. For example, for a web application, a server had to be purchased, a web server had to be built, and the web application had to be placed on that web server.

PaaS (Platform as a Service)

With PaaS, a web server is available in the cloud, so there is no need to buy a server and build it from scratch. You only need to prepare the application or program that runs on the server.

Representative PaaS examples include Google App Engine and Microsoft Azure.

IaaS (Infrastructure as a Service)

IaaS provides infrastructure such as virtual servers, hard disks, and firewalls.

Unlike PaaS, IaaS offers more freedom and allows users to choose hardware specifications, operating systems, and other components. However, it requires knowledge of operating systems, hardware, and networks, and security policies must also be considered.

Representative IaaS examples include Google Compute Engine and Amazon Elastic Compute Cloud.

GCP (Google Cloud Platform)

kc@example.com (kc kim) — Sat, 03 Dec 2022 07:21:00 +0900

What is GCP?

GCP stands for Google Cloud Platform and is the collective name for cloud services provided by Google. Services such as Google Search, Google Maps, Gmail, and YouTube also run on GCP. Originally, services were provided separately, such as Google App Engine, a PaaS service released in 2008, BigQuery, a big data analytics service released in 2011, and Google Compute Engine, an IaaS service released in 2012. In 2013, these services were integrated and provided as Google Cloud Platform.

By using technologies and infrastructure actually used inside Google, efficient web development and operations are possible. Advanced technologies such as data analysis and machine learning, which are also Google strengths, can also be used.

What can you do with GCP?

Because GCP is a cloud service, you can use Google’s advanced technology while reducing introduction and operation costs, without preparing machines, networks, infrastructure, or development platforms yourself. You can do many things with GCP, but the following are representative examples.

Demand forecasting

GCP is characterized by excellent AI and data processing technologies. Customer data can be used for demand forecasting.

Data operations

BigQuery, provided by GCP, is a tool that simplifies data analysis and operations. It can greatly reduce the time and effort required to prepare data analysis and improve operational efficiency. BigQuery can also integrate with Google Analytics 4, enabling Google Analytics data to be analyzed through BigQuery.

Environment that can handle sudden load

GCP provides an environment that can withstand sudden load increases. Even when load rises sharply, it can distribute load and autoscale much faster than many other services.

Game development

Using App Engine, which is fully managed and has excellent autoscaling performance, makes game development possible with small teams and low cost.

Image classification

With AutoML, even without expertise in machine learning, users can create highly accurate models using AI technology simply by setting required items intuitively.

Changing the way work is done

Using Google Workspace can improve work efficiency and also change communication methods.

Benefits of using GCP

In addition to development using Google’s technology, GCP has several benefits in cost and convenience.

Costs can be reduced.

First, GCP has the advantage of relatively low cost. This benefit is easy to understand when comparing GCP with similar services such as AWS and Azure.

Services using the latest technologies are available.

GCP also provides services that use cutting-edge technologies for data analysis and machine learning. Similar services exist in AWS and Azure, but GCP services are known for high performance.

Work can be made more efficient.

Finally, Google handles massive amounts of data every day and can use that experience to perform data analysis efficiently. It is especially characterized by fast data processing, enabling analysis with speed.

Services provided by GCP

Computing

Compute Engine

Compute Engine is an IaaS service provided by Google that provides virtual machines in the cloud. Infrastructure environments such as virtual machines and networks can be used in the cloud, reducing deployment and operation costs.

App Engine

App Engine is a PaaS service provided by Google that provides a platform for running applications in the cloud. It enables app development in a stable environment provided by Google, requires no maintenance, and lets developers focus on development.

Storage

Cloud Storage

Cloud Storage is a highly available object storage service. It has unlimited capacity and automatic backup features, so it can be used not only as a place to store files but also to publish them on the web.

Cloud Datastore

Cloud Datastore is a fully managed NoSQL database. It can automatically scale as needed, enabling high-load processing to run quickly. NoSQL refers to databases that do not use the SQL language and are suitable for handling big data.

Big data

BigQuery

BigQuery is a service for analyzing big data in the cloud at high speed. With BigQuery, 250 million transaction records can be processed in about two and a half minutes, enabling real-time user data analysis.

Cloud Dataflow

Cloud Dataflow is a fully managed processing service that simplifies stream processing for massive real-time data and batch processing. It provides a programming model that integrates a wide range of processing patterns from data acquisition through transformation, analysis, and classification, reducing operational work such as resource management and performance optimization.

Services

Cloud DNS

Cloud DNS is a service that provides reliable and high-performance DNS from Google. DNS converts IP addresses into human-readable strings and is essential for domain management. Cloud DNS can be used to create DNS records.

Translate API

Translate API is an API that can detect and translate strings received through an API regardless of language. When integrated with Google Translate, it can instantly translate between thousands of languages, making multilingual communication easier.

Difference between GCP and AWS

A cloud service often compared with GCP is AWS, provided by Amazon. Like GCP, AWS provides a stable environment and is used by many companies.

There are two major differences between these cloud services.

GCP’s advantage is that it can use Google’s infrastructure. In particular, its services for big data analysis and machine learning using the latest technologies are strengths compared with other cloud services.

AWS, on the other hand, provides a rich set of services and enables flexible web development, including complex system construction. However, because there are so many services, design can easily become complex and specialized knowledge is required.

GCP is strong in data analysis, while AWS is flexible and broad in use. Since each has different strengths, it is best to choose based on the use case.

AWS (Amazon Web Services)

kc@example.com (kc kim) — Tue, 25 May 2021 08:48:27 +0900

EC2 (Elastic Compute Cloud)

A representative Amazon service.
EC2 is a service that rents independent computers to users.
It lets users build as many virtual servers as they need and manage security, network configuration, and storage.
Instances can be turned on and off as needed, and users pay only for the time used.

EC2 pricing policies

On-Demand: Pay for computing power measured by hour or second depending on the running instance.
- No commitment. Used when you want to try EC2 or use it flexibly.
Spot Instance: Buy and use spare market capacity cheaply through an auction-like method.
- Up to about 90% cheaper, but you do not know when it must be returned. Useful when start and stop are flexible or when additional computing power is needed.
Reserved Instance (RI): Commit in advance for a fixed period, usually one to three years, similar to a savings plan.
- Up to about 75% cheaper. Used when demand can be predicted and a commitment period is acceptable for cost savings.
```
Usually On-Demand and Spot Instance are used.
```

EC2 instance types

General purpose: Balanced use of all resources.
- t2: Low-cost general purpose. Web servers and databases.
- m2: General purpose. Application servers.
Compute optimized: Uses high-performance processors.
- c5: Compute optimized. Applications and databases where CPU performance matters.
- F1: Hardware acceleration. Genetic research, financial analysis, and big data analysis.
Memory optimized: Handles large-scale data.
- r4: Memory optimized. Applications and databases where memory performance matters.
- x1e: Memory optimized. Spark.
- p3: Graphics optimized. Machine learning and Bitcoin.
Storage optimized: Handles high disk throughput.
- h1: Disk throughput optimized. Hadoop and MapReduce.
- i3: Disk speed optimized. NoSQL and data warehouses.
- d2: Disk optimized. File servers, data warehouses, and Hadoop.

EBS (Elastic Block Store)

EBS provides persistent block storage volumes for EC2 instances.
Simply put, EBS can be considered a hard disk. Use EBS when an instance running on AWS needs disk space.
Capacity can be purchased as needed and preserved permanently until the user deletes it.
Because data is replicated internally in real time, the probability of losing data is said to be much lower than with a hard disk.
In simple terms, EBS can be attached to EC2 and used like a USB drive.

AMI (Amazon Machine Image)

An AMI can be thought of as a template image containing all software information needed to create an instance.
It is an image with an OS, applications, server program settings, and similar items preconfigured.
When starting an EC2 instance, using an AMI means there is no need to separately install the OS or configure server software.

Three AMI types

AWS AMI: An AMI provided by AWS. It is the default AMI shown when configuring an AMI during instance launch.
AWS Marketplace: A method where software companies create and sell AMIs. Use this type when a company uses specific software.
AWS Custom AMI: The user creates an AMI and launches servers with that AMI.

For example, an EC2 instance currently running in production can be made into a custom AMI so another EC2 instance with the same environment can be started quickly.

ELB (Elastic Load Balancer)

ELB automatically distributes incoming traffic across multiple targets such as EC2 instances, containers, and IP addresses in two or more availability zones.
In other words, it distributes load appropriately, manages servers so they do not fail, automatically detects instance status, and excludes faulty instances from distribution targets.

EIP (Elastic IP)

An EC2 instance server does not have a fixed IP by default.
When an instance runs, it receives a public IP address. If the instance is stopped or becomes unused, the IP address disappears. When it starts again, it receives a new IP address.

The reason a new IP address is received each time is that usable IP addresses are limited. About four billion IP addresses can be used, which means only about four billion users can use the internet. Therefore, IP addresses are reassigned each time.

Because receiving a new IP address each time makes service operation difficult, Elastic IP is used to solve this. Elastic IP assigns a fixed IP address. The IP remains fixed even when unused. However, charges apply if the Elastic IP is allocated but not attached to an EC2 instance. This is because holding an Elastic IP without using it reduces opportunities for others to use it.

Auto Scaling

Auto Scaling is a service that automatically increases or decreases system capacity.

For example, when users gather on a service, CPU usage of participating servers rises. When CPU usage exceeds 80%, an instance is created using a prepared image (AMI), connected to ELB, and traffic is distributed to the newly created EC2 instance.

When traffic decreases and CPU usage falls below 20%, EC2 instances are removed sequentially.

VPC (Virtual Private Cloud)

VPC is a virtual network dedicated to a user’s AWS account.
In cloud services such as AWS, many instances physically run on the same network or equipment, and VPC logically separates them completely.
Networks can be configured by VPC, and each VPC can have different network settings.

VPN (Virtual Private Network)

VPN is a service that creates a highly secure private network connecting computers distributed across a large organization, or connects remote networks over the internet with encryption for more stable and secure communication.

For example, remote workers use VPN to access office networks and files over the internet as if they were working in the office.

IAM (Identity and Access Manager)

IAM is an authorization solution that can grant individual permissions for all AWS solution services and resources.

CloudTrail

CloudTrail records activity in an AWS account.
It records who did what in AWS.
CloudTrail can record, monitor, and retain operations related to account activity in AWS infrastructure, and logs are stored in S3.

CloudWatch

CloudWatch focuses on AWS service and resource activity and reports status and performance.
CloudWatch is a monitoring service for AWS resources and applications.

Route 53

Computers communicate with numeric IP addresses. Because IP addresses are not easy to remember, DNS assigns domain names to IP addresses so they are easier to remember. Route 53 is Amazon’s DNS service.

The core functions of Route 53 are acting as a domain registrar and renting name servers.

CloudFront

CloudFront is a CDN service provided by AWS.
It acts as cache and a Content Delivery Network, aiming to provide faster transfer speeds to users through caching.
It provides efficient content delivery through Edge Locations distributed worldwide.
Edge Locations are collections of cache servers for CloudFront.
Origin Server
- The server that holds the original data.
  - In AWS, origin servers are usually S3 or EC2 instances.
Edge Server = Edge Location
- Servers distributed worldwide and provided by AWS.
- Edge servers provide caching so the same requested data can be returned quickly.

A user accesses a website and requests image and HTML files.
DNS routes the request to an Edge Location that can respond as quickly as possible.
The Edge Location checks whether the requested file exists in its cache.
If the file is not in the cache, the request is sent to the origin server.
The origin server sends the file back to the Edge Location, and the file is added to the cache.
The Edge Location sends the file to the user.

Security Group

A security group acts as a virtual firewall that controls inbound and outbound traffic for instances.
This service can allow a user to connect to a database while working remotely.

References

How CloudFront delivers content