{"id":68,"date":"2015-11-18T23:22:48","date_gmt":"2015-11-18T22:22:48","guid":{"rendered":"http:\/\/188.166.119.205\/?p=68"},"modified":"2026-11-10T05:52:03","modified_gmt":"2026-11-10T05:52:03","slug":"about-the-unix-philosophy-and-why-i-broke-it-and-then-how-i-moved-back-to-the-old-track","status":"publish","type":"post","link":"https:\/\/banyek.net\/?p=68&lang=en","title":{"rendered":"“About the unix philosophy, and why I broke it \u2013 and then how I moved back to the old track”"},"content":{"rendered":"

I am always fascinated by the cleanliness of UNIX. One tool only should do one thing, but it has to be the best in that way. The operating system itself will glue all the modules together and give you a complex feel of a system, you don\u2019t have to take care of huge, bloated software, don\u2019t have to deal with mysterious bugs, which are appearing random. Just small bricks of clever software and the rest are on you.<\/p>\n

Recently I broke this, and frankly, I am not sure if it was a bad decision, or not.<\/p>\n

<\/p>\n

With MySQL, if you want a point in time recovery after a disaster, you should back up not only the database but the binary logs themselves, as I have mentioned it in my last blog post. I added a feature to the binlogstreamer<\/a>: it can clean up the binary logs after a given amount of time.<\/p>\n

Let\u2019s see how many ways we have to get rid of the old, unwanted (expired) files! We can define it in the provisioning tool to \u201censure that directory x<\/em> doesn\u2019t contain files older than y<\/em> days\u201d, like puppet\u2019s tidy module. This is a good solution, clean, and easy to read, but I am not sure if the puppet is the best way to keep the logic of a running system. We can also define a recurring cronjob, for cleaning up a directory (for example with a nicely parameterized find command) which is good enough, moreover this is the UNIXish way but adds complexity, which is avoidable when possible. So what is the \u201cgood\u201d way? I think the best would be the cronjob, but I didn\u2019t choose that.<\/p>\n

The database backups are created with an ansible playbook here, it initializes the mysql host where the backup will be made as well as the storage server where the backups will be kept until they are expired. Because this is a playbook, I found that is a good idea to keep the cleanup process next to the backup one, \u2019cause if the backup process fails I definitely<\/em> don\u2019t want to remove any of the old backups until it is fixed.<\/p>\n

Well, I have two backup processes regarding the database hosts, one does the backup of the database itself, and one starts a binlog streamer process which keeps binlogs saved as well. Because the backup process manages the expiration of the database dumps, it seemed logical to me to make the binlog streamer smart enough to expire binlogs as well. So I\u2019ve decided to add the functionality of cleanup to the binlogstreamer app itself.<\/p>\n

And now I started to rethink the whole idea, and I just wondering if it is not a bit better to move the binlog expiration inside the backup process \u2013 because if the backups are failing, and we keep the old backups to ensure that at last, we have backups to restore from, then we also need binlogs next to them, and I definitely<\/em> don\u2019t want to expire them, until I removed the original backup file too.<\/p>\n

Keep it simple, stupid.<\/p>\n","protected":false},"excerpt":{"rendered":"

I am always fascinated by the cleanliness of UNIX. One tool only should do one thing, but it has to be the best in that way. The operating system itself will glue all the modules together and give you a complex feel of a system, you don\u2019t have to take care of huge, bloated software, […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[134],"tags":[126,208],"class_list":["post-68","post","type-post","status-publish","format-standard","hentry","category-tech-en","tag-oldblog-en","tag-unix-en"],"_links":{"self":[{"href":"https:\/\/banyek.net\/index.php?rest_route=\/wp\/v2\/posts\/68","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/banyek.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/banyek.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/banyek.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/banyek.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=68"}],"version-history":[{"count":1,"href":"https:\/\/banyek.net\/index.php?rest_route=\/wp\/v2\/posts\/68\/revisions"}],"predecessor-version":[{"id":423,"href":"https:\/\/banyek.net\/index.php?rest_route=\/wp\/v2\/posts\/68\/revisions\/423"}],"wp:attachment":[{"href":"https:\/\/banyek.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=68"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/banyek.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=68"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/banyek.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=68"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}