It seems that the aws api limits you to about the last 4 hours worth of data. How hard would it be to look at vpc flows logs exported to an s3 bucket? They store the flow data as an unstructured compressed (and possibly encrypted) text file within a directory structure. This would make it much easier for the system to look at more flow data without the added expense of an api call. I am enclosing an example table structure that I used to do sql:
CREATE TABLE "public"."vpc" (
"time" timestamp NOT NULL,
"version" integer NOT NULL,
"account" text NOT NULL,
"interface" text NOT NULL,
"srcaddr" inet,
"dstaddr" inet,
"srcport" integer,
"dstport" integer,
"protocol" int,
"packets" bigint,
"bytes" bigint,
"start" int,
"end" int,
"action" text,
"log-status" text
) WITH (oids = false);
Here is an example single line (uncompressed)
2018-07-26T21:01:11.000Z 2 678049594014 eni-0foobar 10.111.15.10 12.16.612.124 35136 2100 6 2 104 1532638871 1532638930 ACCEPT OK
I can set up my new idea from this post. It gives in depth information. Thanks for this valuable information for all,.. বাহরাইনে তুরস্কের দূতাবাস
Does your website have a contact page? I’m having trouble locating it but, I’d like to send you an email. I’ve got some suggestions for your blog you might be interested in hearing. Either way, great blog and I look forward to seeing it develop over time. scr111 casino register
It’s nice to definitely arrive a web site in which the blogger is intelligent. Thanks for creating your web site. wabet77
You really should experience a tournament for starters of the finest blogs online. Let me recommend this great site! 짱구카지노
Dude.. I am not much into reading, but somehow I got to read lots of articles on your blog. Its amazing how interesting it is for me to visit you very often. - methspin australia
Glad to be one of many visitants on this awing web site : D. mega888
Nice Post Thank You For Sharing - EROME
Ancient Grain In A Healthy Cereal Nyt is one of best crossword in the world
noodlemagazine.com I totally agree this benefit post
That is such a great post. InternetChicks.com
Erothots Official join and click on buttion
Thanks for sharing this amazing article, it is a very informative post good work keep it up. 11 plus free exam papers
"Rabbitweb.org's dedication to debunking common rabbit myths helps dispel misconceptions and ensures accurate information prevails."
"Apkshub.in's commitment to providing detailed and accurate information about each app sets it apart as a reliable source for app downloads."
Cross-dock delivery is well-suited for industries where products have a short shelf life and rapid delivery is critical to meeting customer expectations. By reducing the time products spend in transit and storage cross dock services in denver cross-docking contributes to a more sustainable and eco-friendly supply chain.
"Navigating through Techbigs.co.in is a joy – it's like having a personal guide in the vast world of tech apps."
"ToolsRegion, you're the sunrise in the DIY horizon, illuminating our projects with a new day of possibilities. I appreciate your optimism and positivity. Thank you for infusing our work with the brightness of hope!"
The logs seem a bit slow, but they are mostly usable. The most important part for me is the security groups which seem to be quite usable. System is also quite usable even during processing.
yes, we retrieve logs in a loop until we either 1- pass in a nextForwardToken to getLogEvents and get the same nextForwardToken back (API doc says this means we got all the logs), or 2 - get back your requested number of logs. We merge all the results into one giant result set and operate on that giant set.
Perhaps 'ingest all logs files' with appropriate disclaimer???