This is an introduction to the ICU websites: how they work, what's involved, how to make changes. It's primarily intended as a resource for ICU webmasters. The websites have evolved since the first one appeared in 2004 and will probably continue to do so in the future. Future webmasters are encouraged to keep this document up to date.
The ICU currently (Feb 2015) runs two web sites:
These websites are managed by a team, one of which is the webmaster. Some examples of roles other than webmaster on the main site are:
And on the ratings site:
When anyone with a role logs in, certain pages become available through which changes can be made to the databases (depending what the role is; see Can-Can for more details). For example:
Further details of tasks that users with roles can accomplish via the websites can be found in the main Help page on this site and in the articles on the ratings site.
The webmaster's role (Admin) is the most powerful one and it enables the webmaster to do anything other users can do (and more). However, the day-to-day content management of the sites (news, articles, ratings, etc) is not the webmaster's top priority. Their chief responsibilities are:
In practice, as well as looking after the technical side of things, the webmaster acts as cover for the rest of the team and can expect to perform a little bit of all the other roles.
The main technical subsystems supporting the two websites, all of which the webmaster is responsible for maintaining, are as follows:
The local environment is what the webmaster has on their own machine to develop, test and deploy new code to the production server. New webmasters should spend some time setting it up correctly as it's essential. It's used to:
Following are the steps to setup the environment for the first time. It's not required to use the same OS (Centos) as the production server, any Unix-like OS should be fine (e.g. MacOS or any flavour of Linux). You should probably avoid Windows.
You can put the ICU code wherever you like but for the sake of an example
(which will be used throughout this document), we'll assume it all goes into
$ cd ~/Projects/icu_www_app/ $ bundle install $ cd ~/Projects/icu_ratings_app/ $ bundle installwhich will install all the other Gems required by the applications.
$ cd ~/Projects/icu_www_app/ $ bin/rails sYou should now be able to see a development version of the main site at: http://127.0.0.1:3000. Type Control-C to quit.
$ cd ~/Projects/icu_ratings_app/ $ bin/rails sand visit the same address to see the development version of the ratings site.
$ cd ~/Projects/icu_www_app/ $ bin/rake # this will take a few minutes $ cd ~/Projects/icu_ratings_app/ $ bin/rake # this will also take a few minutesIf all these tests should pass (or nearly all, see RSpec ), the environment is working correctly.
The steps to deploy a new release of either of the two web applications to the production server are always the same and rely on RSpec , Git and Capistrano to test, commit and deploy. The same commands are run every time:
$ cd ~/Projects/icu_www_app # or ~/Projects/icu_ratings_app for the ratings site $ bin/rake # runs the entire test suite to check nothing has regressed $ git add . # stages all changes $ git commit -m "Comment describing the change" # commits to the local repository $ git push # pushes to the master Github repository $ cap production deploy # deploys to the production server
If anything goes wrong with the deployment, it can be rolled back to the previous release with:
bin/cap production deploy:rollback
The above steps require SSH keys to be registered with both Github and the production server (see Usernames, Passwords and API Keys ).
Most changes the webmaster performs are done either via the web site (to update the database) or via The Local Environment (to update the applications). However, there are some things for which logging into the production server is required (see Centos ). Here are a few examples:
The website-related logs files are:
Other useful directories and files on the production server to know about:
This is a list of the all the services (both the ICU's and those of external providers) which require access credentials of one kind or another (username/password or API keys). The actual credentials are kept in an offline file by the webmaster and copied to a small number of trusted individuals for backup. Some of the credentials are also in secrets.yml or database.yml on the production server (see Secrets ).
The unavoidable costs are:
Optional extras, none of which are currently used (in Feb 2015) are:
Other resources helpful to the webmaster are:
This section contains details of some common webmaster tasks.
If a serious problem develops with one of the websites it can be temporarily taken offline by putting the application on the production server into maintenance mode. This is done from The Local Environment by running the following Capistrano task:
$ cd ~/Projects/icu_www_app/ # or ~/Projects/icu_rating_app/ for the ratings site $ bin/cap production deploy:web:disable
In maintenance mode Apache will redirect all requests to a maintenance page which will display the default message:
We're currently offline for maintenance as of [current time]. Sorry for any inconvenience. We'll be back shortly.
The words "maintenance" and "shortly" in this message can be replaced by setting temporary environment variables like this:
$ REASON="a system upgrade" BACK="in a couple of hours" bin/rake production deploy:web:disable
To exit from maintenance mode and make the site accessible again, run:
$ bin/cap production deploy:web:enable
It's possible to allow the webmaster to continue to view the live websites even if they are in maintenance mode. To do this:
This will allow the site to be accessed by requests from that IP only (all others get the maintenance page). To avoid forgetting to return things to normal once repairs are complete, a prominent notice is displayed at the top of every page whenever maintenance mode is on.
Given a season ticket (more details here), say for example, 2tDdxD, then you can check it's validity by starting a console for the www web application:
$ cd ~/Projects/icu_www_app/
$ bin/rails c
>> st = SeasonTicket.new("2tDdxD")
>> st.expires_on
=> "2010-12-31"
>> st.icu_id
=> "1234"
In this particular example the ticket is valid and belongs to ICU ID 1234 but expired at the end of 2010. If you try the same thing with an invalid ticket you'll get some kind of error, for example:
>> st = SeasonTicket.new("rubbish")
>> st.error
=> "invalid season ticket (bad characters)"
Note that for proper decoding of season tickets in the local environment, the same cryptographic entries configured in the production application's secrets.yml file are required in the local version of that file (see Secrets ).
If you don't have the right crypto entires in secrets.yml, and alternative would be to log into the production server and perform the operation there, but remember to start the console in the production environment because the default is development (which the production server lacks):
$ cd /var/apps/www/current $ RAILS_ENV=production bin/rails c
The webmaster can expect to occasionally receive requests (from, for example, the treasurer or selection committees) for detailed information from the databases which is not available on either of the two websites. One option would be to add new functionality to one of the websites to display the requested information, but that could require a lot of work.
Alternatively, for a quick one-off answer, the data can be generated by logging into one or both databases, issuing the appropriate SQL commands and converting the results to CSV. To do this the webmaster must:
As an example of a complex query best handled this way is to list, for all juniors (under 19) who have been an ICU member:
The data for this particular query is spread across several tables in both databases. One possible way to tackle this is as follows.
First, from the www_production database, get a list of juniors who have subscribed at least once:
SELECT P.id FROM players P WHERE
P.dob > date_sub(current_date(), INTERVAL 19 YEAR) AND
EXISTS(
SELECT * FROM items I WHERE
I.player_id = P.id AND
I.status = 'paid' AND
I.description LIKE '%Subscription%');
Save the IDs as a comma-separated list and in the queries below, replace "[IDs]" with this list.
Next, get the ID, name, DOB, email and club data from the players and clubs tables, taking into account that not all players belong to a club.
SELECT P.id, P.first_name, P.last_name, P.dob, P.email, C.name FROM players P LEFT JOIN clubs C ON P.club_id = C.id WHERE P.id IN ([IDs]);
Next, get the last subscribed date.
SELECT MAX(DATE(I.created_at)) FROM players P, items I WHERE I.player_id = P.id AND I.status = 'paid' AND I.description LIKE '%Subscription%' AND P.id IN ([IDs]) GROUP BY P.id;
Next, switch to the ratings_production database and get the last published rating and the publication date for each player in the list.
SELECT MAX(CONCAT(R.list, ',', R.rating)) FROM icu_players P LEFT JOIN icu_ratings R ON P.id = R.icu_id WHERE P.id IN ([IDs]) GROUP BY P.id;
The above results in 3 data sets all with the same number of rows N (where N is the length of the list of IDs) but with varying numbers of columns (6, 1, and 1 respectively) and they need to be combined into one set with N rows and 8 columns while at the same time converting to CSV format. Here are two ways to get the 3 sets of results in CSV format:
In either case, the result is 3 CSV files (say data1.csv, data2.csv and data3.csv) with N rows and 6, 1 and 1 columns. The final step is to stitch them together horizontally to generate 1 file with N rows and 8 columns using the Unix utility paste:
$ paste -d, data{1,2,3}.csv > data.csv
and, voilà, the answer to the original query is in data.csv.
The page you're currently reading is an example of a help page. Unlike articles and news items, help pages are not stored in the database and can't be changed by editors. Instead, they're part of the web application code and need the webmaster to create, edit and deploy them. To create a new help page do the following in The Local Environment :
$ bin/rspec spec/features/help_spec.rb
The steps for updating existing help pages are the same except the first few steps will have already been done.
The procedure to follow is very similar to the above case (see Create and Update Help Pages ) because, like help pages, ICU documentation is saved to the Github repository and therefore changes are managed from The Local Environment and deployed to the production server by Capistrano . The differences are:
$ bin/rspec spec/features/icu_spec.rb
There are currently (Feb 2015) three categories of fees payable on the <em>www</em> website:
The last category, simple, is designed for fees types that have the following properties:
Simple fees, including the creation of new types in that category, can be managed by the treasurer or webmaster via tools on the website (see Configure a New Simple Fee). The foreign tournament rating fee is an example of a simple fee. Another example is a donation.
For more complex categories of fees, special purpose code has to be written to support their management on the website. This has already been done for subscriptions and tournament entries. Unless their requirements change in the future, no further programming effort is necessary.
However, new types of fees that don't fit into any of the existing categories may arise in the future and to support them on the web site will require custom code to be written. To accomplish this the webmaster will need a thorough understanding of how fees and payments are implemented.
An example of a type of fee that would require new code is the payment of license fees for registered coaches (so-called TRG licenses). There are several reasons that the proposed requirements for this type of fee would make it too complex to fit into the simple category:
As this example shows, implementing new categories could take a fair amount of work even though the framework for doing so is already present. Depending on the skills of and time available to the webmaster, it might be preferable to squeeze new types of fees into the simple category by watering down their requirements, as was done for the example detailed in Configure a New Simple Fee.
Keep packages and libraries on which the applications depend up-to-date. This includes:
Due to the way Linux distributions are curated, updating all the system libraries on the production server in one go with:
$ yum update
shouldn't cause any problems. However, updating all Gems in the local environment with
$ bundle update
is a riskier proposition. It may be difficult to identify the cause if a lot of gems get updated but something goes wrong. Where gems are concerned, it's better to update in small steps (a few at a time) and afterwards run the test suite. Big changes introduced by new major versions, especially Rails , may require changes to the applications.
July (or whenever the Irish Championships finish): Update the right hand panel of the home page of the www site with the names and photos of the new men's and women's champions. The file to edit is: app/views/layouts/_features.html.haml.
July: Make sure the Treasurer has rolled over the current set of subscriptions ready to go on sale on the 1st August (see Rollover Subscriptions).
July: Renew the security certificate for www.icu.ie so that https continues to work for credit card payments (see SSL/TLS ).
September (or whenever the executive changes): Update the database records for ICU officers and email addresses (see Manage Officers and Mail Relays).
December: Make sure the annual fee for the icu.ie domain has been paid. It should be automatic as long as a valid credit card is registered with the account (see Register365 ).
The web server is Apache and its configuration is in:
Log files are in:
Log rotation is configured by files in /etc/logrotate.d: www, ratings and httpd.
Apache passes requests for the two websites to the application server (see Passenger ), so these requests get logged twice, firstly in one of the Apache logs and secondly in one of the two application logs:
The backup policy is to regularly (once a day) save data (the databases) and files that are not in Github (images, uploads, tournament flyers, Apache configuration, firewall settings and Secrets ) to a directory (~/bak) on the production server and then periodically rsync this entire directory to one or more offsite locations (for example the webmaster's desktop and laptop). The server backups are managed by the script ~/bin/bak.sh run every night via cron (see Whenever ).
The crontab entry for the rsync running on one of the webmaster's home computers might look something like this:
0 10,14,18,22 * * * cd ~/Projects/bak; rsync -rtvz user@aontas.icu.ie:bak/ aontas > aontas.log 2>&1
Multiple attempts per day are made because home computers are not always turned on.
Since the source code for the Rails applications and ICU Gems are in Git repositories, there's no need to back them up.
Linode offer an inexpensive backup of the entire server which would make it extremely easy to recover from a failure. It's up to each individual webmaster to decide whether to make use of this or not. If you're confident that, if necessary, you could rebuild the server from scratch, you might decide it's not worth it. Currently (Feb 2015) we aren't using it.
Bundler is a tool for managing Gems . Both web applications list the gems they depend on in a file called Gemfile which may also specify constraints on the required gem versions. The explicitly required gems often have implicit dependencies (specified by the gem authors) on yet more gems. Bundler works out a set of gem versions that will satisfy all constraints and saves that set to a file called Gemfile.lock which is committed to the repository, ensuring that all instances of the application will use the exact same set of gems.
In The Local Environment the tool is invoked from the command line. For example:
$ bundle install
will install any gems from Gemfile.lock that are not already installed. On the production server, bundler is automatically invoked by Capistrano during deployment.
Most gems are under active development and new versions are regularly available. If the following command is run locally:
$ bundle update
the latest available versions of all gems (consistent with the explicit constraints in Gemfile and the implicit constraints of the gem authors) will be installed and Gemfile.lock updated. However, caution should be exercised when updating many gems at the same time as it may be difficult to identify the cause of any incompatibilities that may arise. A better approach is to update a small group of gems together and then run the tests before committing to the new versions. For example, to update just the ICU gems that the ratings application depends on:
$ bundle update icu_tournament icu_ratings icu_name
This will also try to update the icu_utils gem which is an implicit dependency of icu_tournament (the dependency on icu_utils is specified in the manifest for the icu_tournament gem, not the rating application's Gemfile).
For more details on bundler, see the website.
Can-can is a gem for authorization and maps User Roles to permissions via the file app/models/ability.rb (www, ratings) along with directives in controllers and views. To give an example from the www site, in ability.rb:
if user.calendar? || user.editor? can :create, Event can [:destroy, :update], Event, user_id: user.id end
means that only users who have the role of Editor or Calendar Editor can create, update or delete calendar events and, further, they can only update or delete events that they created themselves, that is, when the user ID of the event (user_id) matches the user's own ID (user.id).
Capistrano is a command-line tool for automating code deployments. The most common Capistrano task that the webmaster runs is:
$ bin/cap production deploy
which is part of the Test, Commit, Push, Deploy sequence of commands to update one of the websites. This simple command makes many things happen on the production server:
If anything goes wrong with a deployment, it can be rolled back with:
bin/cap production deploy:rollback
To get more detailed feedback, temporarily change the log_level in the file config/deploy.rb to debug.
There are other Capistrano tasks and the following command will list them all.
bin/cap -T
Most are subtasks of the main deploy task. In practice the only tasks the webmaster needs to know about are:
Centos is the Linux distribution used on the production server and is currently (Feb 2015) at version 6.5. A description of configuring Centos and installing additional software to support the websites (Ruby , Apache , MyQSL , etc.) is available for:
There's also a detailed log (available from MO as an Evernote page) of all changes made to aontas between November 2013 (when it was first created) and September 2014, which might be useful to future webmasters.
Maintenance of Centos consists of the webmaster periodically updating the system software using the yum package management tool Keep Software Up-To-Date ). This is important to provide protection against security vulnerabilities such as Heartbleed (April 2014) and Shellshock (September 2014).
Upgrading Centos to the next major version is desirable at some stage although it's not currently (Feb 2015) urgent. Full updates for Centos 6 should be available until 2017 and maintenance updates until 2020 (according to this Wikipedia article). The previous update (Centos 5 to Centos 6, ratalacha to aontas) was managed by temporarily having a second server running the new version of Centos which took over serving the websites when it was fully configured and tested. After that the old server was shut down and the ICU went back to paying Linode for just one server.
This service is for tournament controllers to upload pairing lists, standings and live games and then place links to those on their own websites. Managing the files is entirely up to the controllers (subject to guidelines below), the webmaster just provides the account. People who want to use it should ask the webmaster for the username and password. For security reasons the webmaster should change the password from time to time, but not while any tournaments are in progress (unless urgent).
There are two subdirectories where users can place their files:
Users are encouraged to create a subdirectory in which to place all their files. For example, the files for the Bunratty 2015 tournaments might all go into sm/bunratty15/. This avoids the files for different tournaments getting mixed up together.
Gems are packages of Ruby code designed to perform a particular task. Gems usually come with their own documentation and may have dependencies on other gems. When Ruby is installed, many standard gems are installed with it. Rails (the application framework used by the ICU websites) is a gem itself and has a large number of gem dependencies. Gems are versioned and often under active development with new versions appearing regularly. Bundler is a tool to help application developers establish a set of gem versions that work together.
Gems are stored in repositories the biggest of which is RubyGems. The repositories used are declared at the top of Bundler's Gemfile. The ICU have developed some gems primarily for its websites but which could theoretically be used independently. Two of the gems are specifically for the ratings site:
The other two are for code and data that is shared between the two sites:
The same procedure is used to update any of these ICU gems, including the web applications that use them. Here we illustrate how to make an update to the icu_ratings gem. First, the gem is updated, tested and released:
$ cd ~/Projects/icu_ratings/ # change to the gem's project directory $ vi ... # make changes $ bin/rake # run the entire test suite to detect regressions $ vi lib/icu_ratings/version.rb # update the version number $ bin/rake build # check for build errors $ bin/rake release # push to RubyGems $ git add . # stage and ... $ git commit -m '...' # ... commit changes to the local repository $ git push # push changes to the master repository
Next, the web applications that use the gem (in this case only icu_ratings_app) are updated, tested and deployed:
$ cd ~/Projects/icu_ratings_app/ # change to the application's project directory $ bin/bundle update icu_ratings # update to the latest version of the gem $ bin/rake # run the entire test suite to detect regressions $ git add . # stage and ... $ git commit -m '...' # ... commit changes to the local repository $ git push # push changes to the master repository $ bin/cap production deploy # deploy the update to the production server
Note that the web applications (icu_ww_app, icu_ratings_app) are not gems. Their only public presence are their master Github repositories, whereas the ICU gems are on both Github and RubyGems. During a Capistrano deployment, new application code is downloaded (by Git ) to the production server from Github while new gems are downloaded (by Bundler ) from RubyGems.
Git is a tool for software version control. For each of the ICU web applications and Gems there are 2 Git repositories:
Code changes for both the applications and gems are first committed to the local repository and then pushed to the master. The webmaster needs to install the Git command line tool for The Local Environment and optionally may also wish to use a GUI (such as Tower on MacOS).
Anyone interested in contributing code can fork (copy) the public repositories, add their changes and then send a pull request (proposal to incorporate their work) to the webmaster, the only person with write access to the public repositories. The webmaster would pull the changes into the local repository, test them and decide whether or not to accept them. If accepted they would be committed locally, pushed to the master repository and ultimately deployed to the production server by Capistrano . If rejected, they would be deleted from the local repository and never make it to the master or the production server.
In theory, there could be a webmaster team with one person (the official ICU webmaster) being in charge, merging updates into the master repository and deploying them to the production server. The other team members would have their own local environments but without the ability to update the master repositories or to deploy to the production server.
There are public Github repositories for each of the two web applications:
Plus 4 more for the Gems :
Anyone can read these repositories but only the webmaster can write to them. Secrets are never committed to any repositories, including the public ones.
Google Charts is used for rating graphs and the www site uses Google Maps for club and event maps. The latter requires an API key which goes in config/secrets.yaml (see Secrets ).
Linode is the company that provides the ICU's hosting services. They're a US company with data centers in several locations round the world the nearest one of which to Ireland is in London and that is where the ICU production server is.
The service consists of:
For extra payment, the following are also available:
See also The Cost of the Websites .
HAML is a templating language (HTML with embedded Ruby code) and is the main one used by the ICU web applications. This help page, for example, is written in HAML. Erb, the default Rails templating language, is used in some JavaScript files. To illustrate the difference between HAML and Erb for HTML, here is a simple example. First, HAML:
%p This is a demo.
%ul
%li= "The date is #{ Date.today.to_s(:db) }."
%li= "The number of players is #{ Player.count }."
Second, Erb:
<p>This is a demo.</p> <ul> <li>The date is <%= Date.today.to_s(:db)} %>.</li> <li>The number of players is <%= Player.count %>."</li> </ul>
Both examples produce the same output, namely:
This is a demo.
Instead of running its own mail server, the ICU has chosen to take advantage of one of the mail services that have sprung up in recent years such as AmazonSES, Mailjet, Mandrill, Postmark, SendGrid and Mailgun (which is the one we use). The main benefit of these services for us is that, given the ICU's relatively low monthly transactions, most of them are free. For Mailgun the free threshold is 10,000 transactions per month (which so far, in Feb 2015, has never been exceeded) and the cost for each additional transaction over this limit is only $0.0005. Unless a credit card is registered with the account, then the service simply stops if the threshold is exceeded and no mail would be delivered until the start of the next monthly billing period.
There are two types of email delivery this service provides:
In order to work, the relays need the following (which the webmaster is responsible for):
See Register365 for more on DNS entries and Manage Officers and Mail Relays for how to manage the @icu.ie mappings.
As well as delivering mail, the Mailgun service includes a log of all transactions which is very useful for identifying delivery problems. Sometimes the reason for a failed delivery is right there in the log. As long as a credit card is registered with Mailgun the log includes the last 30 days (without a credit card, just 1 day). It doesn't contain the content of messages but it does include sender, recipients, subject and delivery status. It also includes timestamps of when recipients open emails that were delivered to them.
In practice the Mailgun service is free, but in theory the cost is unlimited (if a credit card is registered). Therefore certain safeguards are in place to automatically disable all relays (which are responsible for most of the transactions) if either the predicted end-of-month cost is too high or there is a sudden spike in transactions. Additionally, details of the predicted cost are calculated on the ICU server every night (using Mailgun's API) and the details sent by email to the webmaster.
If the relays are automatically disabled the webmaster should:
There are two MySQL databases on the production server, one for each web application:
Locally, each of these has development and test counterpart (see The Local Environment ):
The test databases should be empty (test data is loaded automatically when tests run). The development databases should be loaded with recent copies of the production databases.
There are several Rake tasks related to the databases. To see a list of them run:
$ bin/rake -T db
To make changes to either of the databases, the webmaster first creates a migration (see Active Record Migrations) and then runs the following rake command to update the local database:
$ bin/rake db:migrate
During deployment to the production server, database migration is handled automatically by Capistrano .
There are a number of options for serving Ruby applications such as Mongrel, Thin and Unicorn. The choice for the ICU websites is the free version of Phusion Passenger. This software keeps copies of both applications running in memory, even when there are no requests to process, in order to avoid the overhead of restarting applications for every request.
The usual way of installing a new version of Passenger on the production server is to login as root, download the latest Passenger gem and run the command to install the rest of the software:
$ gem install passenger $ passenger-install-apache2-module
The installation software will give directions on how to update Apache configuration (/etc/httpd/conf/httpd.conf) to complete the installation. It also has good diagnostics if there are any installation problems.
The task in lib/capistrano/tasks/restart.rake (in both applications) ensures that Phusion will be restarted after a Capistrano deployment.
Before upgrading Ruby make sure that Passenger supports the new version.
It's not necessary to run Passenger in The Local Environment , although it's certainly possible. WEBrick is sufficient.
Rails is a model-view-controller application framework. It:
Developers are free to choose their own favorite technologies over the defaults, as the ICU have done for:
The Guides and API are amongst the best internet resources for Rails.
Rails is under active development and new versions with new features come out regularly. Just upgrading the Rails gem is usually not enough as often there are changes to be made elsewhere, such as the configuration files:
For this reason, the precise version of the Rails gem currently in use is always specified in Bundler 's Gemfile.
To upgrade from The Local Environment :
It is possible to have different versions of Rails running the two web applications. However, to avoid confusion, if one is upgraded then the other should be too.
The rails command line tool is often used in The Local Environment . Some examples:
$ bin/rails s # start webrick on port 3000 $ bin/rails g ... # generate skeleton files for new code or migrations $ bin/rails db -p # log into the development database
Another useful command is to start an irb (interactive Ruby) console with the development environment already loaded. Any database queries issued are echoed along with and their execution time.
$ bin/rails c >> Rails.env => "development" >> Player.count (10.7ms) SELECT COUNT(*) FROM `players` => 18604
Rake is a Make-like build tool implemented in Ruby with tasks and dependencies specified in standard Ruby syntax. Rails provides several application-related Rake tasks and Capistrano is implemented on top of Rake. Some Rake tasks the webmaster will often run in The Local Environment are:
$ bin/rake db:migrate # migrate the developent database $ bin/rake db:rollback # return the development database to the previous state $ bin/rake routes # examine the routing table $ bin/rake spec # run all spec tests $ bin/rake # run all spec tests (that is the default task) $ bin/rake -T # print a list of all rake tasks
Redis is a fast key-value store used for retrieving translations (www site only) if the user enables the option for displaying parts of the site (headers and links only) in Irish. Translations are stored in the SQL database (to support their management by the translator) and synced to the Redis database. When pages are rendered translations come from the fast Redis store and not the relatively slow SQL database.
Unfortunately, although the technical work to support a partly bi-lingual www site is complete, the work of actually translating all the English language text snippets is not. The original translator, who also inspired the project, quit without completing the project and nobody has come forward to replace her. Therefore the controls to swap between English and Irish are currently (Feb 2015) disabled on the production server (not in development or test) via the method irish_enabled? in app/helpers/application_helper.rb.
The webmaster should install Redis in The Local Environment and start it running with its default port, otherwise parts of the test suite will fail.
This is the ICU's domain provider and through the control panel the webmaster is able to make DNS updates or change the credit card used to make annual payments. Some DNS entries are for the Linode production server and the others are for handling mail via Mailgun . Here's a snapshot taken in Feb 2015:
This is the preferred (over Test::Unit or Minitest) test framework. It's used in conjunction with a number of other Gems :
The tests are all in the directory spec with most in:
Some special directories and files are:
The entire test suite can be run as follows and takes a few minutes:
$ bin/rspec spec
Alternatively, since this is the default Rake command, the following will do exactly the same thing:
$ bin/rake
At the end of any development work the webmaster should always run the entire test suite before committing anything to the local repository. During development, avoid repeatedly running all tests (that would be too slow). Instead, identify a small subset of existing relevant tests that can provide more rapid feedback on whether anything has been broken. Also, strongly consider writing new tests for the feature being developed to defend against future regressions.
An example of a quick, focused test designed to be run repeatedly during development is the test for this very page of documentation:
$ bin/rspec spec/features/help_spec.rb -e overview
It may seem strange to be testing a page of documentation but in fact there is a system of keys being used for the headings and links in this page (to avoid repeating the same text over and over again) and the test is useful to detect spelling mistakes in these keys.
Since Rails is the chosen web application framework, it follows that most of the application code is written in Ruby, an elegant, object-oriented language which surged in popularity around 2005 (partly due to the success of Rails). The language is still in active development.
As with Rails and Passenger the general approach to new versions, at least up to now (Feb 2015), has been to upgrade on a regular basis, if it's safe to do so. With Ruby, the main thing to check is that Passenger and Rails support the new version.
On the production server there is just one version of the ruby executable, the one in /usr/local/bin. The installation steps are illustrated below for the 2.0.0 version:
$ cd ~/src
$ wget http://cache.ruby-lang.org/pub/ruby/2.0/ruby-2.0.0.tar.gz
$ tar -zxvf ruby-2.0.0.tar.gz
$ cd ruby-2.0.0
$ ./configure
$ make
$ sudo make install
$ touch /var/apps/{www,ratings}/current/tmp/restart.txt
The last step restarts both web applications.
In The Local Environment , the webmaster should use the same version of Ruby as on the production server. However, it's recommended to have multiple versions of Ruby installed and managed by either rbenv or rvm. This makes it easy to test both applications with a new version of Ruby before upgrading the version on the production server.
The file .ruby-version in the root directory of both applications has no effect on the production server but, locally, it automatically runs the specified version of Ruby in development or test mode if either rbenv or rvm is installed. So testing a new version is as simple as installing it, setting it in the .ruby-version file (for both applications) and then running both test suites.
Each web application has a pair of files that are used to store sensitive information such as API keys, database connection parameters, cryptographic data and passwords. The two files are:
These files are NEVER COMMITTED TO THE REPOSITORY because anyone can view the Github data. For this reason, the above two files are included in .gitignore for both web apps.
When setting up a development and test environment, separate versions of these files (different to those on the production server) are required. Sample files showing what's required are available in the repository in config/examples/, but they contain dummy data. Examples of data that will differ between live site and local environment are:
A few things can remain the same, such as the API key for Google Maps (see Google Apps ).
A new webmaster, setting up their local environment for the first time should:
See also Usernames, Passwords and API Keys .
Stripe provides a way for individuals and organizations to accept payments over the internet. Although the company is American it was founded by two brothers from Limerick. Previously the ICU used Paypal but Stripe is technically superior and setting up and verifying an account is easier.
As the ICU has no legal standing (not being incorporated) it's necessary for an individual (called a sole trader) to own the Stripe account on behalf of the ICU and for that person's name and address to be used for the account verification process (the same applied to PayPal). Once the account has been verified and connected to the ICU's bank account two pairs of API keys (one pair for production and one pair for test and development) need to be registered with the application (see Secrets and Usernames, Passwords and API Keys ) before payments can be made.
The person who owns the account is normally a member of the ICU committee at the time the account is first setup. They are able to login to the account dashboard (on the Stripe website) which, amongst other things, provides details of all transactions made through the account. Most of the information that the treasurer needs is recorded in the ICU database but, if the account holder is not also the treasurer, it's normal to share the dashboard login credentials with the treasurer so that they can also benefit from the additional source of information.
One issue that the sole trader model gives rise to is what happens when that person leaves the ICU committee and doesn't wish to continue in the role (and may even wish to shut down their Stripe account)? With Stripe the solution is easy:
Note that when a new Stripe account is used then payments made with older accounts can no longer be automatically refunded (if any refunds are required). Instead payments would first have to be revoked in the ICU database and then reimbursement made manually (e.g. sending a cheque) — see Carts, Items, Refunds and the Sales Ledger for details of the distinction between refunds and revokes.
To enable secure communication between the user's browser and the ICU server during credit card purchases there is an SSL (aka TLS) certificate for the www.icu.ie domain which is renewed annually. The first one ever purchased from 123-reg (the cheapest that could be found at the time) cost £10 and expired on 4th August 2015, so the right time to start the renewal process each year is July (see Seasonal Tasks ).
Notes:
There is support in the www application for automatically switching into secure mode, when required, and back out of it, when no longer required. Thus the user will stay in normal mode unless they go to login or make a credit card payment. See the helper methods switch_to_tls and switch_from_tls in app/controllers/application_controller.rb and the file app/assets/javascripts/switch_from_tls.js.
Other than a common reliance on jQuery (via the jquery-rails gem) the two web applications take different approaches to CSS and JavaScript extensions.
The www site uses Twitter Bootstrap, a responsive framework (styling adapts to the size of the viewing device). Moreover, signed-in members can choose from a number of themes (different versions of Bootstrap provided by Bootswatch). Some of the implementation details:
From time to time new bug-fix versions of Bootstrap (in the version 3 series) are released and Bootswatch usually follows suit shortly thereafter. The webmaster should replace the old Bootswatch files with the new versions and then go through the usual Test, Commit, Push, Deploy procedure.
The ratings site is more basic: there are no user-selectable themes and it's not responsive. Styling is provided by a custom set of rules written in SASS, a developer-friendly extension of CSS, in the file app/assets/stylesheets/main.sass. This file is automatically compiled to CSS by the Rails Asset Pipeline..
Support for dynamic features is provided by jQuery UI. The Dialog widget, in particular, is used a lot in the administrative part of the site. The hover-to-reveal-menu behaviour of the top navigation bar is due to something called SuperFish (see vendor/assets/javascripts/superfish.js).
Webalizer is a simple web server log file analysis program. It's reports for the two sites cab be viewed here: www, ratings.
The Webalizer configuration files for the two sites are:
Unlike most other scheduled tasks (see Whenever ), which run under the ordinary user, the two Webalizer daily tasks are scheduled in the root user's crontab.
Run from an application's root directory in The Local Environment , the command:
$ bin/rails s
("s" is short for "server") starts a small web server called Webrick (distributed as part of Ruby's standard library) running on the default port 3000. The application can then be accessed in development mode at: http://localhost:3000.
There's no particular need to replicate the setup on the production server (Apache plus Passenger ), Webrick is perfectly adequate for development work. The converse, however, would not be true: Webrick would be too slow for the production websites.
Both applications and the production server have tasks that need to be scheduled with crontab.
For www:
For ratings:
For the production server:
Managing these tasks could be achieved by requiring the webmaster to (a) log into the production server and (b) manually edit the crontab file. However, a better solution is to make use of the Whenever gem and configure application specific tasks in the file config/schedule.rb. There are two benefits:
To illustrate, compare the following Whenever task specification in config/schedule.rb:
every :day, at: "5:15am" do rake "pgn:db" end
with the corresponding specification in crontab:
15 5 * * * cd /var/apps/www/releases/current && RAILS_ENV=production bin/rake pgn:db >> log/cron.log
That said, there are some tasks that are entered directly in the crontab and not managed by Whenever (because either they're not application specific or need to run under root) and those are Backups and Webalizer .