Resource Management

As a certified Google Cloud Architect, when you’re thinking about how you want to design your organization’s infrastructure, you must think about how the design can continue to grow and evolve in the future so that you’re designing an organizational hierarchy that enables your company to adapt to rapid change. Turning your infrastructure into code modules massively enables this. But the way your environments are all provisioned also plays a big role in the culture of your enterprise. Your goal is to release your developers from their limitations in the cloud and enable them to think and build freely and innovate in the most unique ways, while also protecting the security and safety of your cloud environment. If you’re designing your organization hierarchy now, it is an exciting time for your company.

Cloud Resource Manager Overview
Google Cloud Resource Manager is a mechanism that provides resource containers such as organizations, folders, and projects and enables you to group and hierarchically organize GCP resources into those containers. With the Cloud Resource Manager API, you are also able to programmatically manage these resource containers. Having the ability to manage all of your projects centrally is an important function for the administrators of the cloud. With the Cloud Resource Manager, you can create an organization that contains all of your projects and resources within those projects. You can also create folders to group your projects according to teams, business units, environment, products, and so on. IAM policies can also be applied at the organization and folder levels. If you think about this structure, you’ll realize that it enables you to set access control policies (ACLs) at various levels that will propagate to all of the child resources below. Lastly, the Cloud Resource Manager gives you the ability to programmatically manage all of this, including all of the change history, so that you can identify recently deleted projects and changes and, if necessary, undo those changes.

Organization Hierarchy
How you design and set up your organization hierarchy is quite an important aspect of a well-architected cloud environment. As companies begin to move to the cloud, some of the technical debt that was accumulated from the various changes and growth stages in on-premises technology during the past 30 years may look like a California wildfire zone: California being dry is only one reason why these massive wildfires continue to happen. When trees and brush dry out and die due to lack of water, they remain in forests, where they slowly decompose, creating fuel for future fires. Natural wildfires are apparently good for the environment, because they return trapped nutrients back to the soil and remove disease-ridden plants from an ecosystem. But you can read more about that in a science book. The moral of our cloud story is this: although it might be okay for dead trees to remain in a forest to fuel future fires and nurture the earth, we want to avoid fires in our cloud environments. If you think of technical debts as dead trees in your cloud forest, you realize that at some point, these debts may be responsible for turning a small event into a major incident. The word fire is something that all technologists hope never to hear in the workplace.
Imagine how many remnants of an application or design may remain in your on-premises environment because people thought that the infrastructure was needed elsewhere, they didn’t know where it belonged, or they were just afraid to delete it because it could accidentally take down the entire business. Staying organized enables security and governance teams to have full visibility over what’s going on in their enterprise so that they are able to manage change, decommission and remove unnecessary bits of technology, prevent internal users from making mistakes, and prevent malicious or incompetent users from doing bad things. That’s one compelling aspect of staying organized. Another one is that with the move to the cloud, companies must adopt a culture of rapid innovation and growth. Can you imagine being tasked with redesigning your living room if your roommates were hoarding junk? That’s not an environment that promotes innovation and growth. Like the Marie Kondo philosophy that promotes organizing your environment to improve your well-being and mental performance, your goal for developers is to eliminate all potential distractions so they can think clearly, try bold ideas, and rapidly deploy changes while minimizing the risks to reliability and security. That being said, let’s talk about the organization hierarchy in GCP.

TIP: Remember the two major elements of the resource hierarchy in GCP: it provides a hierarchy of ownership that binds your resource life cycle to its immediate parent in the resource hierarchy, and it provides points to attach and inherit access control and organization policies.

Organization, Folders, Projects, and Resources
Let’s talk about what this resource hierarchy looks like. You need to know the capabilities of each level of the hierarchy, the typical uses for each level, and the importance of your naming conventions. Companies use a variety of design patterns for their organizational hierarchies.

Resources
Resources are the lowest level of components that make up GCP services. Think about a virtual machine (VM), a GCS bucket, a container, and even pub/sub (publish/subscribe) topics: these are all resources. Resources can be parented only by projects, not by other resources, and each resource has only one parent. That way, when you apply any policies to a parent, it is inherited by all the child resources. So when viewing billing or usage data, for example, you can dive into the lowest common denominator, your resources, to discover what costs the most or what is driving up your metrics.

Projects
Resources roll up into projects, which are the first grouping mechanism for the resource hierarchy. Projects can contain multiple resources. As mentioned, ACLs, VMs, APIs, and other cloud resources can all be grouped into projects. Projects are not tied to one another, and it’s important to group related resources into your projects, both from a functional and access control standpoint. Projects provide a point to attach ACLs, and you can have as many projects as you want without any associated costs. Each project’s logical grouping is completely independent of other projects.

You are required to use projects on GCP. It provides the foundation to create, enable, and use all GCP services, manage APIs, do billing, and manage permissions. There are a few key components of a project:
- Two identifiers:
- A project ID, which is a unique ID for your project
- A project number, which is a read-only number that gets automatically assigned upon project creation
- One mutable display name
- The life cycle state of the project (ACTIVE or DELETE_REQUESTED)
- Any labels that are attached to it
- The time of its creation

Every time you interact with a resource, you’ll need to identify the project info for every single request you make. You can use either the projectId or the projectNumber field. By default, the creator of the project is granted the owner role for any newly created projects.

It’s important that you know the difference between a mutable and immutable object. Mutable objects can be changed after they’ve been created. Immutable objects cannot be changed after they’ve been created. An example of a mutable object is the project resource; you can change it as you please after it’s created. An example of an immutable object is a file that is uploaded to Google Cloud Storage; once you upload the file, it cannot be changed throughout its storage lifetime.

Folders
Projects roll up into folders, which are additional logical containers that are intended to group similar projects. You want to group similar projects inside your folders so that you can apply a level of consistency to the policies you attach to your folders. While it’s easy to think about what makes the most logical sense based on the way your business is organized, it’s best for you to think about designing your GCP organization hierarchy in a manner that is aligned with how you want to separate your access patterns or how you apply your policies. Suppose you have a bunch of VMs that run a highly sensitive workload, grouped into a project, which is then grouped into a single folder that propagates the ACLs to all developers. Since that folder contains all development projects, you’re not following the principles of separation of duties and least privilege. You can have up to ten levels of folders, so it’s common that companies would typically start with one or two and work their way up to as many as they need. Think about how you’d like to organize your folders—by business units, applications, environments, or products. Sometimes even creating a separate folder for your control plane can be a pattern as well.
You need to have appropriate permissions to view any folder resources. Folders act as a policy inheritance point for identity and access management (IAM) and organization policies. So if you grant an IAM role to a folder, it’ll be automatically inherited by subfolders and all projects underneath it.
 

Organization
Folders roll up to an organization, which is the root node and top-level element of this whole hierarchy. All resources that belong to an organization are grouped under the organization node, enabling companies to have central visibility and control over all the resources that apply to the organization. An organization can represent a company, and any IAM policy that applies to that organization will propagate throughout the entire hierarchy beneath it. When your first user with a Google Workspace or Cloud Identity account creates a project, an organization is automatically provisioned alongside it.

Some GCP services require that an organization be created before they’re allowed to be leveraged. Shared VPC is an example of this.
Having an organization resource is necessary to manage resource life cycles. It’s also important in identity management, recovering organizations in the event of an account recovery, and giving all your resources one central point of ownership. With an organization, there is always a mechanism to track accountability for resources in Google Cloud. Imagine, for example, that an employee who created an important project leaves your company. What happens to the user’s project when her account is deprovisioned and deleted? Imagine that the employee’s projects were deleted. That would be a sight to see. A better idea is when an employee leaves your company, their projects follow the organization’s life cycle policies. Take a look at Figure 4-1for a visual representation of this resource hierarchy.

Sample resource hierarchy in GCP
Suppose there is a major US bank called BankyBank. Let’s say Product 1 is BankyBank’s flagship mobile banking app. Team A is the Data Science team, Team B is the Mobile Banking dev team, Credit Card users are Dept X, and Dept Y is the Retail Banking department. From an IAM perspective, you can get an idea here of how access is managed:
- The most obvious point, users who do not have access to the organization will not have access to anything within the organization.
- Credit Card (Dept X) users will not have inherent access to the Retail Banking (Dept Y) resources.
- BankyBank’s Data Science team (Team A) does not have inherent access to the resources of the Mobile Banking team (Team B).
- BankyBank’s flagship mobile banking application is in a separate folder (Product 1) that is PCI-compliant, and their back-office applications are in a separate folder (Product 2). These are two separate groups of developers.
- Development, test, and production support teams are all prevented from accessing one another’s environments by default. It just so happens, however, that the dev/test teams share the same IAM policy, so those users can access both the dev and test projects. We’ve also not applied any nonprogrammatic IAM access to the production environments, because there is an approval process in place to request break-glass access when a production issue managed by the support team occurs. A. you know, with regard to the concept of defense-in-depth, least privileges, and separation of duties, this organization hierarchy now enables your business to have a multifaceted approach to how it designs, operates, and secures its applications. By not allowing any users to access the production environments, we’ve already eliminated the possibility of a nonprivileged user compromise from potentially harming the business. BankyBank also does not use live production data in its dev/test environments. So if a developer were to be pwned (aka compromised) in the dev or test environment, much of the risk has been minimized. Now you can imagine that when users get promoted or move to different teams, their privileges must be removed to eliminate privilege creep. Can you imagine what happens when a long-tenured employee who has played a pivotal role across the enterprise and accumulated permissions across all the businesses gets pwned? Not good!
The IAM policy hierarchy follows the path of the resource hierarchy. If you make changes to your resource hierarchy, the policy hierarchy will also change. Here’s an example of this: If you move a project between folders, it will change the inherited permissions. If those permissions are from a different parent, they will be lost when that project is moved. Once you move that project into another folder, it inherits everything from the parent hierarchy.

Organization Policies
Organization policies are a configuration of constraints that are set on a resource hierarchy node (at the project, folder, or organization level). These organization policies enforce the restrictions defined in the organization’s policy and enforce the restrictions on each resource and all of its child resources. To define and set the policy, you need to have the organization policy administrator role. When you choose a constraint, you can configure that constraint against a set of desired restrictions (run this constraint on a specific folder). All the child resources inherit that organization policy. If you apply an organization policy to your organization node, you’ve basically enforced this constraint across your entire organization.
Constraints can be either list constraints or Boolean constraints. List constraints evaluate the constraints with a list of allow/deny values that you provide. An example of this is whitelisting IP addresses that can connect to a VM. Boolean constraints can govern specific behaviors via a true/false restriction. For example, if you want to run the disable service account creation constraint within a folder, you’d set the Boolean to true to prevent users within that folder from creating service accounts.
It’s also important to know that when an organization policy is applied, it is not retroactive. If you set an organization policy in a folder to restrict a certain behavior that is already occurring, you’ll see that your policy is in violation, but it will not stop that behavior from happening. You’ll need to remediate that behavior manually. Imagine, for example, that your organization decides it no longer wants to allow users to leverage CloudSQL and deploys this in an organization policy without giving its users ample time to migrate from CloudSQL—that could be disastrous! In such cases, you won’t be able to use organization policies as a mechanism to stop behaviors from preexisting resources; you’ll need to remediate those yourself when you see the retroactive violations.

Best Practices
When you’re building a cloud environment for your enterprise, there are several considerations to think about as you’re defining this organization structure—how you’re going to automate your project creation, what naming convention will be associated with your resource containers, whether you’ll use many projects or few projects, and so on. It’s much easier to do your due diligence at the beginning and try to predict what the future of your business will look like, instead of going through an entire rework a few years down the road. The cloud is agile, though, and if you follow infrastructure as code (IaC) as a best practice, it won’t be too difficult to manage when things change (as they should) in your organization’s future.
Considerations with regard to how you’ll be separating resources into projects include the environment that your projects will be operating in, the roles that will be assigned to users within the project (aligned to a strong separation of duties model), how you plan on doing billing, some of the quotas and limits that will be tracked by your projects, and what could go wrong if a project were to be compromised. As a best practice, you should use a many projects approach, rather than few, as you’ll end up doing the heavy work up-front. This will serve you well from a change management, access, and configuration standpoint in the future. If you automate your project factory, it will make your life even easier. For example, you probably would not want to design a structure with just three projects that are for each environment (development/test/production). Instead, use a hierarchy similar to what was described earlier, creating projects for each one of your applications in each environment. This will make governing your access model a lot easier. It will also make it a lot easier to label projects properly, to identify what’s costing you the most money, and to minimize the amount of harm if a single project were to be exposed. If you follow the basic security principles in the design of your cloud, you will be very successful!
Lastly, there is the concept of a project lien, which is a function that will protect your projects against project deletion. This is important, especially for your production environments. You don’t want someone accidentally deleting your project and shutting your business down due to a clerical error!

Exercise: Creating a Project
1. Head to Cloud IAM and ensure you have the resourcemanager.projects.create permission, which is included in the roles/resourcemanager.projectCreator role.
2. Open the Cloud Console and go to the Manage Resources page.
3. Select the organization for which you want to create a project. If you’re a free trial user, this list will not appear, so, you can skip this step.
4. Click Create Project.
5. In the New Project window, enter a project name and select an appropriate billing account.
6. In the Location field, enter the name of the folder in which you want to add the project.
7. When you’re finished, click Create.

Additional References
If you’d like more information about the topics discussed in this chapter, check out these sources:
- Cloud Resource Manager Concepts https://cloud.google.com/resource-manager/docs/concepts
- Google Cloud Platform Resource Management www.youtube.com/watch?v=MzclA_hdNLY