Setting up AWS web page publishing

For my future reference1 I shall describe how to set up the command-line magic needed at my Linux PC to upload from it to the AWS "cloud" all my 'molehole' static HTML web pages that constitute my external web site.

Here's a neat picture...

... showing how AWS hosts 'molehole' for me:

How AWS hosts molehole

I'm sure Mr Bezos won't mind me borrowing it. After all, I've been an Amazon customer since 1999. And after several months of daily use I'm delighted by how well the system works.


... simply by having a son who already rents a storage "bucket" on AWS. Actually, I also have just such a bucket — I set it up in the immediate wake of Yet Another Windows PC Collapse in early 2008 to safely contain some precious photos of Christa — but since I've left it untouched for eight years I hate to disturb its slumber. (AWS ticked me off some months ago when they noticed I had not "rotated its keys" for seven years!)

We need an encrypted storage box

We began by installing and testing dropbox (he did that before I could tell him it was already installed, of course — I just hadn't bothered to use it for several months):

sudo apt-get install dropbox
sudo apt-get autoremove
dropbox start -i
cd Dropbox/
open .
start .
touch foobar
rm foobar 

So now we have somewhere to keep the super-strong password, in encrypted form, that he insisted I use to keep his AWS access credentials tucked up safe in bed.

A password manager

How about a password manager, Dad? Why don't you use one instead of writing them all down? (I hesitate to suggest that if a miscreant has physical access to my PC I have more worrying problems since the proportion of my "digital Life" is but a miniscule fraction of his.) So choose "keepass", install it from the Mint distro, notice it's way back-level, and fix that while we're about it:

sudo apt-add-repository ppa:jtaylor/keepass -y
sudo apt-get update
sudo apt-get install keepass2 -y

Some way of getting stuff into keepass!

Clipboard Management? It was while trying to copy and paste this ghastly password into the "keepass" facility that we now ran full tilt into the brick wall I've mentioned — it turns out Linux has two buffers for some of its copy and paste operations. Which one a given copy or paste operation actually uses remains a Mystery, but certainly explains some odd inconsistencies I've noticed as a Linux neophyte. Anyway, these next tools sort that out, though they also now park a Clipboard Manager icon in the System Tray and, if I sneak up on it, I can see his super-strong password in clear text which strikes me as a bit "iffy" somehow. Still, that's easily cleared (and probably equally easily restored, but let's keep going).

sudo apt-get install xdotool
sudo apt-get install autocutsel
autocutsel -s PRIMARY &
sudo apt-get install parcellite

A hidden folder for AWS credentials

He's popped three files in here: config, credentials, and s3_website.yml — all containing simple plain text bits of magic associated with allowing my userid to access his AWS admin console (which, having seen, I frankly hope never to have to use2) and the AWS storage bucket assigned for 'molehole' pages to splosh around in. He has also reserved the right to move this elsewhere, but assures me he will do that "invisibly" as far as my simple use case is concerned.

mkdir .aws
chmod 700 .aws
cd .aws
vim config
vim credentials
chmod 600 credentials 

The AWS command-line tool

A bit of an anti-climax, if you ask me:

sudo pip install awscli

Followed by some initial web page file uploading:

aws s3 sync <local path> s3://<bucket>/<path>/

A musical interlude

At this point, the bash_history file reveals that I fired off a "get_iplayer" request to download the performance of the Philip Glass opera "Ahknaten" that I had missed while all this downloading and a whole slew of uploading attempts had been going on.

Creating a "publish-aws" script for me

He rightly suspects my ability to type accurately, clearly.

cd bin
ls -al
chmod +x publish-aws 
ls -al
echo $PATH
cd ..
ls -al
rm core

Notice the way he quietly removed a core dump without so much as a murmur.

All over...

... bar the shouting. I shall draw a discreet veil over a series of painful attempts to get AWStats working, with sidebars involving Ruby on Rails, Java, and some other gorp. I did, however, put my foot down and refuse to let him put Apache on just for web stats, and he proved unequal to the task of arranging to download the AWS weblogs and then integrating AWStats (as my reporting package) with lighttpd (my locally-running webserver).

He also tried to get cloudfront working, which (I gather) would have dispersed 'molehole' to the four corners of the world, but could have raised some occasionally awkward image-cacheing issues as it sends out asynchronous strings of "refresh" commands hither and yon. Since I'm not running some vast, commercial, DB-backend, web site it all seemed like overkill.

So, as far as I know, 'molehole' is now being served up as simple, fast, static, HTML from a single AWS storage bucket in (either) the west of Ireland (or) Arizona. Thanks, son! Think of the money you'll save on ISP fees.


1  A code phrase meaning "next time I do something silly and have to re-install all these processes from scratch".
2  Not least because it also entailed installing an App called "Authy" on my smartphone, registering and syncing that with some aspect of the AWS console (a process that actually saw me scanning one of those abstract squares of digital data that performed some sort of Masonic secret handshake with "Authy" before deciding we were good to go) and then being given 30 seconds each time to supply to the AWS console a "challenge" numeric code sent to my phone to prove I was in physical possession of the thing whenever wanting to access the AWS console...