VPC Atlas AWS 網絡清單新增可篩選的 CSV 格式

What the VPC Atlas inventory holds, the four formats it comes in, and why I made the CSV one flat row per resource.

The diagram is the picture, the inventory is the list

A diagram tells you what a network looks like. It's a poor answer to "which subnets are public in eu-west-1", to "what changed since last month" or to "send me every VPC with its CIDRs". You can't search a picture, and nobody diffs one by eye with any confidence. For those questions I wanted a list: one document with everything the scan found, that a person can filter and a script can read.

VPC Atlas already wrote that document as JSON, Markdown and HTML. Version 0.7.0 adds a fourth rendering, a CSV. This post covers what the inventory holds, who each format is for, how to get it, three ways to filter it, and the choices I made in the CSV, including the one that puts a stray apostrophe in your cells.

What the document holds

The inventory is the account written as a tree. Under each audited account come its regions. Under a region come its VPCs, plus the resources that sit in no VPC, such as Transit Gateways and peering connections. A VPC holds its subnets and the resources that sit in no subnet, such as route tables and security groups. A subnet holds what's placed in it, such as NAT gateways and load balancers, and carries its Availability Zone, its CIDR and a tier: public when the default route goes to an internet gateway, otherwise private, isolated or unknown.

Next to the network tree, each account has a service layer in five families: DNS and edge, API and load balancing, compute and containers, data and storage, and account services such as CloudTrail and GuardDuty. When the scan couldn't read a value in the account services, the file says not read. That's different from zero or no, so a denied call never reads as an empty account.

Then come the parts that cross accounts. Accounts the scan saw but has no credentials for are listed as external accounts, every link that crosses an account boundary has its own entry, and the calls that failed while discovering are listed as issues. A summary with the counts opens the document. Every resource carries the same fields: id, name, ARN, state, its details, its tags, whether the owner is an account you didn't audit, and which audited accounts saw it.

Four formats, four readers

JSON, in vpcatlas-inventory.json, is for machines and for jq. Its schema id is vpcatlas-inventory/1. Markdown is for a pull request or a wiki: the same document as tables, with the details cut at 240 characters per cell. HTML is a single self-contained page with a search box that filters rows by id, name, type or account, and it's the one I'd send to someone who won't open a terminal. CSV is for a spreadsheet or csvkit.

They're four renderings of one document, and the same graph always gives the same document.

How to get it

vpcatlas discover writes the inventory with the diagrams. vpcatlas inventory scans the same way but draws nothing, which is what you want for an audit attachment or a scheduled job:

vpcatlas inventory --profile prod --format json --output-dir out

--format takes json, md, html, csv or all, repeated or separated with commas, and all is the four. The files land next to vpcatlas-graph.json in the output directory, vpcatlas-out by default.

The dashboard serves the same four at /export/inventory.json, /export/inventory.md, /export/inventory.html and /export/inventory.csv, and the inventory tab has a .csv button. /export/all.zip holds the lot: the diagrams, the inventory in the four formats, and graph.json.

If you already have a saved graph, you don't need AWS at all:

vpcatlas inventory --cache ./out/vpcatlas-graph.json --format csv

Three ways to filter it

In a spreadsheet, turn on the filter on the header row and pick a value in the type column, say subnet, then a region in the region column. That's all it takes to get the subnets of one region.

With csvkit, the same question as a command, from the docs:

csvgrep -c type -m subnet vpcatlas-out/vpcatlas-inventory.csv | csvcut -c region,vpc_id,id,name,cidr

With jq, on the JSON, here are the public subnets, one per line with account, region, VPC, subnet, CIDR, zone and tier. This is the recipe from the agent skill's references with one select added, and INV is the path of the JSON file:

INV=vpcatlas-out/vpcatlas-inventory.json
jq -r '.accounts[] as $a | $a.regions[] as $r | $r.vpcs[] as $v | $v.subnets[]
  | select(.tier == "public")
  | [$a.account_id, $r.region, $v.vpc.id, .subnet.id, .cidr, .availability_zone, .tier] | @tsv' "$INV"

Why the CSV looks the way it does

The first decision was the shape. The JSON is a tree, and a tree doesn't fit in a table. The tidy answer looked like one file per level: accounts, VPCs, subnets, each pointing at its parent by id. I didn't build that, because the first thing anyone does with a spreadsheet is filter, and a filter works on one sheet. With three files, "everything in this subnet" turns into a lookup between files, which is the work I wanted the file to save you.

So it's one table with a row per resource and a scope column that says where the row sits: region, vpc, subnet or service. The VPC and subnet ids are repeated on every row under them, so a row still makes sense after you've filtered the rest away. The price is 23 columns. Rows come in document order, a subnet row and then the resources in it. For a resource of the service layer, family says which one it belongs to.

Details and tags are the opposite problem. A subnet and a NAT gateway don't share attributes, so a column for each would give a very wide table, mostly empty. Both are written as key=value; key=value, in document order, and unlike the Markdown file the details aren't cut.

The cell guard came from what's in the cells. Tags are text that anyone with access to the account can write, and a spreadsheet will run a cell that starts with =. So a cell starting with =, +, - or @ gets a leading apostrophe, which makes the spreadsheet show the text instead of running it. The full-width forms of those signs count too, and a tab or carriage return in front of one is removed first, so it can't hide the sign. The trade-off is that a cell starting with a minus sign gets the apostrophe as well, harmless or not. The apostrophe is part of the file. Strip it if you read the CSV in code.

The rest is smaller. Lines end with CRLF, as the CSV format has it. The header row is always written, even when the scan found no resource, so a script that reads the file by column name never meets an empty file. A missing value is an empty cell, and the same document gives the same bytes, so two CSVs of the same graph compare equal. The file is UTF-8 with no byte order mark: if a spreadsheet shows accents wrongly, import the file and choose UTF-8. A spreadsheet may also read a 12-digit account id as a number and drop its leading zeros, so import that column as text when you need the exact id.

What it doesn't do

• The CSV has the resources and nothing else. External accounts, cross-account links and issues aren't rows; they're in the JSON and the Markdown.
• The number of rows can differ from summary.resources in the JSON. A VPC that was referenced but never described in the scan has a placeholder row in the tree, and the summary doesn't count it.
• A scan can only name what its credentials can see. VPC Atlas doesn't discover the accounts of an AWS Organization yet: you list them yourself, with --profile or --role-arn repeated. When it runs from the management account, it reads Organizations only to put names on accounts.
• Each file is one scan, with the time of that scan in generated_at. It's a snapshot, not a history. To compare two moments you keep two graph files, or let the dashboard record the scans and draw the difference.

Tell me what's missing

The format reference is in the inventory docs, the command in the inventory page, and the code in the repository. If there's a column you needed and didn't find, or a filter you ended up writing by hand, tell me which one.