{"id":27,"date":"2019-05-01T04:18:44","date_gmt":"2019-05-01T09:18:44","guid":{"rendered":"https:\/\/codeblock.supadupa.rocks\/?p=27"},"modified":"2022-06-18T15:24:14","modified_gmt":"2022-06-18T20:24:14","slug":"awk","status":"publish","type":"post","link":"https:\/\/client35.dev\/code\/awk\/","title":{"rendered":"Awk"},"content":{"rendered":"Reading Time: <\/span> 2<\/span> minutes<\/span><\/span>\n

Awk is a powerful tool that we’ll take a look at to parse our logs. It’s much more powerful than what we’re going to be using it for in these tutorials, but I encourage you to learn more about awk on your own if you like it.

Awk separates a file into columns (known as fields) and rows – whenever you think of a file and how awk parses it, think of columns and rows.

With this knowledge, we can now take a look at awk’s -F<\/code> flag which allows us to specify how awk sees columns. By default, awk uses a space as its delimiter to read through text. In the following example, we can see that if we just print<\/code> then awk is going to print the entire file:
<\/p>\n\n\n\n

awk '{ print }' \/var\/log\/nginx\/site.access.log| head -n3\n\n100.43.90.123 supadupa.rocks - [27\/Apr\/2019:00:54:32 +0000] "GET \/robots.txt HTTP\/1.1" 200 106 "-" "Mozilla\/5.0 (compatible; YandexBot\/3.0; +http:\/\/yandex.com\/bots)"\n127.0.0.1 supadupa.rocks - [27\/Apr\/2019:00:59:52 +0000] "GET \/\/?wp-cmd=ping HTTP\/1.0" 200 5 "-" "curl\/7.47.0"\n127.0.0.1 supadupa.rocks - [27\/Apr\/2019:00:59:53 +0000] "GET \/\/wp-admin\/?wp-cmd=ensure HTTP\/1.0" 200 305 "-" "curl\/7.47.0"<\/pre><\/div>\n\n\n\n
Now if we go to print field $1 (the IP addresses) we simply need – 
awk '{ print $1 }' \/var\/log\/nginx\/site.apachestyle.log<\/code> <\/p>\n\n\n\n
awk '{ print $1 }' \/var\/log\/nginx\/site.access.log |head -n3\n100.43.90.123\n127.0.0.1\n127.0.0.1<\/pre><\/div>\n\n\n\n
If I wanted to search like grep, but using awk, there’s a solution for that. With awk, you have to put your regex nested between two \/<\/code> in order for it to be interpreted. 
Additionally, if your regex has special characters inside of it (like ?!*&<\/code> or other other special characters) you will need to escape them using a \\<\/code>. See awk’s man page for more information. 
<\/p>\n\n\n\n
awk -F'"' '\/104\\.196\\.177\\.220\/{ print $2 }' \/var\/log\/nginx\/site.apachestyle.log |head -n3\nPOST \/wp-cron.php?doing_wp_cron=1556326791.3251020908355712890625 HTTP\/1.0\nPOST \/wp-cron.php?doing_wp_cron=1556334044.2906210422515869140625 HTTP\/1.0\nPOST \/wp-cron.php?doing_wp_cron=1556336718.0919361114501953125000 HTTP\/1.0<\/pre><\/div>\n\n\n\n
That’s all for now. I hope you enjoyed reading this post!<\/p>\n","protected":false},"excerpt":{"rendered":"
Reading Time: <\/span>  2<\/span> minutes<\/span><\/span>Awk is a powerful tool that we’ll take a look at to parse our logs. It’s much more powerful than what we’re going to be using it for in these tutorials, but I encourage you to learn more about awk on your own if you like it. Awk separates a file into columns (known as […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"footnotes":""},"categories":[2],"tags":[],"class_list":["post-27","post","type-post","status-publish","format-standard","hentry","category-command-line-tutorials"],"acf":[],"_links":{"self":[{"href":"https:\/\/client35.dev\/code\/wp-json\/wp\/v2\/posts\/27","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/client35.dev\/code\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/client35.dev\/code\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/client35.dev\/code\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/client35.dev\/code\/wp-json\/wp\/v2\/comments?post=27"}],"version-history":[{"count":0,"href":"https:\/\/client35.dev\/code\/wp-json\/wp\/v2\/posts\/27\/revisions"}],"wp:attachment":[{"href":"https:\/\/client35.dev\/code\/wp-json\/wp\/v2\/media?parent=27"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/client35.dev\/code\/wp-json\/wp\/v2\/categories?post=27"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/client35.dev\/code\/wp-json\/wp\/v2\/tags?post=27"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}