The AWS Project Development Kit (PDK) and projen, which it’s built on, can save you so much time during project setup. As I’m sure you know, it is daunting to get all of the elements of a fully functional TypeScript project up and running:
- Is the target Node.js, browsers, or Deno?
- What TypeScript version, settings, strictness level, etc.?
- Separate TypeScript settings for the tools and testing?
- Jest configuration?
- Linter and formatted settings?
- IDE configuration?
- Support ESM?
The effort to get all of that configured and setup, then put it in a Git repo and set up pipelines for package maintenance, lookup sample code for the frameworks you’re using in the project, and then you have to keep that up to date! And if you have multiple projects (and everyone does) to maintain, then can all be very overwhelming!
Fear not! PDK and projen are the solution for you! They use the same Construct architecture of the CDK (Cloud Development Kit) line of frameworks: AWS Cloud Development Kit (CDK), CDKtf (for Terraform), and CDK8s (for Kubernetes). As a bonus, since it’s based on the same jsii core, it also gains the polyglot abilities to manage Python and Java projects as well, complete with configuration written in Javascript, TypeScript, Java, Python, or JSON.
Perspective: Project-as-Code
Configuration management and infrastructure management have evolved a lot over the years, in particular over the last decade. I’ll spare you the history lesson, but its worth highlighting a few things we’ve learned:
- Configuration is tedious, as is anything that represents a complex graph
- Configuration is not static, but instead needs to have flexibility while maintaining repeatability
- For this reasons it is best to manage infrastructure as a declarative and parameterized configuration, with idempotent execution
- Declarative and parameterized configuration has a lot of boilerplate, redundancy, and logic that is best handled with a language that includes branching, looping, and composability
These explain the evolution to solutions such as AWS CDK (Cloud Development Kit) and HashiCorp Terraform where you describe (or declare) the infrastructure state that you want with a programming language and some parameters, and the tooling (through various stages) executes whatever CRUD operations as needed to accomplish that described target state. As mentioned before, there’s also CDKtf, which is CDK for Terraform, and CDK8s which is CDK for Kubernetes, that provide these capabilities for those systems.
This description of configuration in code is called configuration-as-code (CaC), and when the configuration is that of an infrastructure, it’s called infrastructure-as-code (IaC).
Patterns of usage emerge when using tools like these. These patterns can be encoded into components in both of these systems, in Terraform these are called modules, and in the various CDKs they are called constructs. The components can then be further composed, made into components themselves, etc.
What projen provides is this same declarative configuration-as-code and composition to projects, where we define a project as a folder of code and config files that are used to build a usable artifact such as a library or service. This is project-as-code (PaC). And the PDK further adds some tooling and modules for things like monorepo management and automatic graph-generation of CDK projects.
In all of these cases, there’s a couple of additional and important side-effects of using components in this way: they can evolve, and everything that uses those modules evolves with them, and they can be inspected mechanically. This means adjustments upstream, such as new features or best-practice corrections, are brought in with updates. And having the components be inspectable yields tools like cdk-nag which provides security and best-practice enforcement, and cdk-graph-plugin-diagram that reads the infrastructure graph and makes a diagram of the target state.
Before You Start
A few notes about projen and we’ll get into putting together a configured (opinionated) project in minutes, and then imposing our own difference of opinion onto it:
- You’ll need Node.js installed, even if you’re making a Python or Java project.
- Along with Node the tools npm (Node Package Manager) and npx (Node Package eXecute) are installed, both are actually part of the npm package. We’ll be using them both extensively.
- Whether you use npm, or alternatives such as yarn and pnpm, is a personal preference.
- We will stick with npm to reduce the global dependency load (so far we only have the Node manager [fnm or nvm] as a global dependency).
- All commands in this blog assume a bash-like shell, but should work or be easily converted to PowerShell for Windows use.
Bootstrap Your Project with PDK and projen
The first step of a new project is to bootstrap it. This is usually done manually or with a tool like yeoman or create-next-app. However, those tools stop there, where projen doesn’t.
Note: Do NOT run this in an existing project. It will not work as expected. It is possible to convert a project to projen but that’s beyond the scope of this article.
Assuming you have fnm (or nvm) installed, the commands you would run to make a new project would be:
Since the pdk docs suggest a different command for calling pdk after installing it globally:
- I suggest avoid installing anything globally unless it’s absolutely necessary (see ‘Why is it better to use a Node manager?’ above).
- npm and npx are installed as part of the node installation.
- Since the pdk command is not in a package called “pdk“, we have to use the package name @aws/pdk instead.
- Adding –yes before the package name tells it we acknowledge that it might need to downloaded, don’t bother asking for permission first.
Now you have a fully configured TypeScript monorepo base project! A lot just happened with that last command:
- npx installed the pdk npm package in a cache directory, then executed it to install projen and it’s dependencies in the current folder, which was then executed to build the rest of the files.
- npm was used because we passed –package-manager=npm instead of the default yarn.
- A .projenrc.ts file was created to house all of the configuration we’ve given it so far, as well as any we’ll add.
- Any time .projerc.ts is modified, run npx projen to have it run and recreate all managed files.
- It’s a TypeScript file since that was the default. Add –projenrc-py to the command to make a Python file, or –projenrc-java to make a Java file, the project will still be a monorepo-ts project either way.
- Since we passed –eslint=true –prettier=true, ESLint and Prettier were installed and configured with opinionated defaults.
- The files tsconfig.json and tsconfig.dev.json were created with opinionated defaults.
- GitHub configuration and workflows were created to lint pull requests, build pull requests, and routinely update dependencies.
- Set up package.json with a slew of ready-to-use scripts, including build, test, watch, eslint, and projen.
- Installed and configured nx to manage the monorepo and handle running tests and builds in all the subprojects with a single command.
- Once all the files were made, a git project was initialized in current folder with a managed .gitignore file and an initial commit was made.
Note that there are other types of projects to choose from. We chose monorepo-ts for this example. This allows us to build and maintain multiple types of project in the same parent project. Execute npx projen new –help to get a list of available project types.
Manage Your New Project
One thing you may have noticed while browsing the project files is that some files have notes in them about being created by projen and that they aren’t to be edited. These are files you would usually maintain manually, such as package.json and tsconfig.json. You will now modify them indirectly by changing the .projenrc.ts and running npx projen to regenerate all of the controlled files. This allows us to impose changes to those files while letting projen handle the details, including updating those files over time when projen updates.
+ Add an App
Now we want to add a TypeScript App as a sub-project. Normally, we would add dependencies at this point, but since this type of project is built-into projen, we don’t need to add any new dependencies yet.
Now we’ll modify .projenrc.ts to contain the following code:
Save the file and then run npx projen to execute the .projenrc.ts and you should now have a ready-to-run TypeScript project in ./app .
What we did was:
- Added a new import so we can get to the typescript.TypeScriptAppProject definition in the projen module.
- Added a new project with a parent of project – which was already defined as our monorepo.MonorepoTsProject.
- Set the outdir attribute to put the new TypeScript app in the ./app directory.
- Gave the sub-project a name and a default release branch.
- Set packageManager: javascript.NodePackageManager.NPM as it must match that of the monorepo, and yarn is the default (which we’re not using because we’re avoiding global installs).
- Since this was not the initial pdk or projen call the changes were not automatically added to git. You’re on your own managing git not.
- Note that some of the tooling assumes you’re using conventional commits, and uses the commits in a build to determine when to bump release version numbers.
The npm run command will list all of the options of built-in commands. If we’re inside ./app we’ll get different options, specific to that project and that type of project. (Hint: If using VSCode, to get linting, formatting, etc. to work as intended, run code -a ./app
to add the ./app
folder to your workspace.)
Note that you now have two projects (in projen lingo): the parent monorepo project and the child TypeScript App project, but you still only have one .projenrc.ts file that configures it all.
For example, you can run all the tests in all of your subprojects by running npm test.
+ Add a package
If you are following a tutorial or are following instructions in a ReadMe and see instructions that say run npm install –save package1 package2 or run yarn install package1 package2 then add the following to .projenrc.ts (before project.synth()):
project.addDeps("package1", "package2");
Or modify the typescript.TypeScriptAppProject properties to add to the deps array.
For dev dependencies, use:
project.addDevDeps("package1", "package2");
Or modify the typescript.TypeScriptAppProject properties to add to the devDeps array.
For –save-peer, use:
project.addPeerDeps("package1", "package2");
Or modify the typescript.TypeScriptAppProject properties to add to the peerDeps array.
Remember that any time you change the .projenrc.ts file, you will need to run npx projen to update the project.
+ Update the Configs
You can override the contents of any config file projen generates by changing .projenrc.ts. This includes the tsconfig.json (used for the app code) and tsconfig.dev.json (used for the tooling, including interpreting .projenrc.ts ) files.
Remember, projen is opinionated, and as such it will impose some defaults. Everything can be overridden, but the more you adjust from the defaults, the more likely you will break stuff. Luckily, when you bootstrapped, projen also initialized a git repo, so as long as you commit (or just stage changes) before making any possible breaking changes you can still roll them back easily. And in this case, the projen tools themselves may fail to run once the tsconfig files have been modified. In that case, roll the tsconfig.dev.json file back and projen should run again.
In this example, our opinion would be to extend the @tsconfig/node18 package to control the TypeScript configuration and then override from there. As an example of how to do this, replace the typescript.TypeScriptAppProject definition in .projenrc.ts with the following:
Now run npx projen to update the project with those changes, and you should see that you have a new dependency in package.json and the tsconfig.json and tsconfig.dev.json files both changes the be much smaller and concise. You may also notice that “moduleResolution” was set to “bundler” – this is an example of overriding what’s in @tsconfig/node18.
Many of those attributes are undefined since projen defaults have value for those attributes and the configurations are deep-merged. Passing in an undefined value indicates that instead of merging those attributes you’d prefer that they be removed.
Create a Custom Project Type
In a monorepo, we can put new project types in the ./src folder and refer to them from .projenrs.ts. Here we’ll encode all of the changes to make a TypeScript app that uses @tsconfig/node18 as the basis for TypeScript configuration, ESM modules and top-level-await support, and uses esbuild to bundle the ./app/src/index.ts (and all of it’s dependencies) into a single Javascript file (and a source map for debugging, of course).
Make a new ./src folder at the top of the monorepo (not in the app/ directory) if it doesn’t exist, and copy the tms-typescript-app-project.ts to it, then change the .projenrc.ts to the following:
Delete the ./app folder and then run npx projen to build a new one with those changes. (If you don’t delete the ./app folder your changes in there will be preserved, but you might not get the built-in sample files that use top-level-await from the new project type.)
Where to Go from Here
PDK and projen are both very powerful tools, and allow you to have full programmatic and repeatable control of your projects. One of the most powerful capabilities would be to make external libraries of base project types your company uses, allowing you to effectively maintain the base config of all of the projects that use those libraries from one place.
We have published an NPM module @10mi2/tms-projen-projects (also on GitHub) with some of our project types, such as a TypeScript App project that is preconfigured for ESM and bundling with esbuild. We’ll be updating it with new project types, and pull-requests are welcome!