Version 0.4.1
Due to significant changes to the Dreamhost setup, please upgrade to version 0.4.1 as soon as possible.
Dreamhost PS Manager is an open source tool that dynamically manages the size of the memory of your Dreamhost virtual private server.
Managing the size of your memory dynamically, has a certain number of advantages. The main advantage is that you can continue to serve your visitors during peak traffic periods by temporarily increasing available resources without having to pay the monthly fee for these resources, instead you are only charged for the time the resources were made available to you. Conversely during low traffic periods your resources will be lowered saving you from having to pay for resources that you do not really need all the time. Basically this puts a 4000 MB RAM webserver at your disposal at a fraction of the price.
It also saves you from having to guess how much memory you need and from having to set your server's memory size manually. Dreamhost PS Manager will do that for you based on actual usage.
You will need an API key for the Dreamhost web panel.
You can get one here:
https://panel.dreamhost.com/?tree=home.api.
This API key needs to be allowed to access dreamhost_ps-set_size
and dreamhost_ps-list_ps.
Note: If you are upgrading please check that your API key can access both these items.
Proceed to download the PsManager script and the configuration file:
Place both of these files in the same directory, for example in a "bin" directory of any user on the private server you want to manage. Avoid placing this script in your cgi-bin directory or under your site's document root, it should not be accessible via your website.
You can install PsManager directly from the command line of your VPS:
mkdir ~/bin 2>/dev/null; cd ~/bin wget http://www.7is7.com/software/dreamhost/download/psmanager.tar.gz tar -zxvf psmanager.tar.gz
If you did not follow the procedure above, check that the permissions are
in order: psmanager.pl must be executable and
psmanager.conf only readable by the owner (or type:
chmod 700 psmanager.pl; chmod 600 psmanager.conf).
Note: Is it important that your configuration file is not readable by others at it contains your Dreamhost API key and may contain your Twitter or Identi.ca passwords.
When the above is done, edit the configuration file filling in your Dreamhost webpanel username and API key. Optionally you can also set the userid and password for a Twitter or Identica account (see below for more about that).
$username = 'me@mysite.com'; $key = 'ABCDEFGHIKLM';
$microblog = 'Identica'; # 'Identica' or 'Twitter' $uname = 'my_server'; $pass = 'secret';
Finally to get everything running, you'll need to set up your crontab to run psmanager.pl every minute (this example assumes the program is located in your "bin" directory):
* * * * * $HOME/bin/psmanager.pl > /tmp/psmanager_cron.log 2>&1
After collecting a minimum amount of data, PsManager will start to make decisions about what memory size your system needs.
When upgrading from an older version, one generally only needs to
replace psmanager.pl, as the configuration file can usually
stay the same. You can basically follow the procedure above without replacing
your original psmanager.conf file
The instruction below will only extract the perl program from the tar-file after downloading it:
cd ~/bin wget http://www.7is7.com/software/dreamhost/download/psmanager.tar.gz tar -zxvf psmanager.tar.gz psmanager.pl
By default Dreamhost Ps Manager will set a memory level between 300 MB and 4000 MB, which are respectively the minimum and maximum allowed by Dreamhost. You can set a higher minimum or lower maximum in the configuration file (the lines are commented out by default).
A sudden increase of popularity of your site (either legitimate or the result of foul play) can cause an increase in allocated memory and higher hosting costs. To avoid the risk of allocating memory beyond what you are prepared to pay for it is recommended to change the maximum memory setting to a level that you feel comfortable with. For example:
$mem_max = 2000;
You can also create more (or less) space by applying a memory factor other than 1 (which is the default). For example if you want to add 10% to the calculated memory, set memory_factor = 1.1.
$memory_factor = 1.1;
In February and March 2010 Dreamhost significantly changed it's memory configuration. A major change is that caching is now reported separately instead of being included in the total used memory. Cache memory is mainly used to speed up disk access (often accessed files are kept in memory). However cached memory is always available for processes when needed, and so it's really freely available memory.
If you want to reserve more memory for caching purposes you could consider to increase the allocated memory by a more significant factor. This is not needed for the proper functioning of your webserver but it can't hurt either and will speed up your server a bit. For example to double the allocated memory:
$memory_factor = 2;
Version 0.4.0 was released when Dreamhost changed the memory setting yet again in March 2010. Since that time no extra memory got allocated, instead you had to set the total you wanted. The extra buffer we were given before for free disappeared, but the price of memory was halved at the same time. To compensate for the loss of this buffer, we now maintain a buffer of 50% by default to preserve burstability of server. However this can be changed; lower it if you want to live dangerously and risk running out of memory or raise it if you want more peace of mind, though we think 50% (the default) is about right.
$buffer = 50; # Memory buffer in percent
PsManager should be able to determine the server you want to manage automatically, if for some reason it doesn't you can manually set the 'ps' variable in the configuration file to indicate the correct machine.
$ps = 'ps0000';
As of version 0.2.4 you can set complex Perl instructions in the configuration file, for example to increase the minimum required memory at certain times of the day. For example the code below will set the minimum memory to 400MB from 7pm (19:00) to midnight.
{
my ($sec,$min,$hour) = localtime();
if ($hour>=19 && $hour<=23) {
$mem_min = 400;
$memory_factor = 1;
}
}
A really cool feature is undoubtedly that you can follow changes to your Dreamhost PS server via Twitter or Identica. The idea is simple, you create a Twitter or Identi.ca account for your Dreamhost PS and let PsManager update it. You then proceed to follow this account from your own Twitter or Identi.ca account. Each time the memory of your Dreamhost PS changes you'll see a tweet about the change appear in your updates stream.
You can check how PsManager tweets the changes it makes to my Dreamhost PS:
As of version 0.3.3 it is possible to define your own template for the tweet, by adding a line to the configuration file, for example:
$tweet_template = "{change} {ps} resized to {newsize} MB from {oldsize} MB at {time}";
Where:
{ps} = the name of your VPS
{newsize} = the new memory allocation in MB
{oldsize} = the old memory allocation in MB
{time} = the time at which this change occurred
{change} = A visual representation of the change with - and + signs:
- = decrease of < 100 MB
+ = increase of < 100 MB
-- = decrease of < 200 MB
++ = increase of < 200 MB
--- = decrease of < 300 MB
+++ = increase of < 300 MB
etc...
If you are using the Dreamhost PS Manager script, you may also be interested in using the Dreamhost PS Rebooter script, to automatically schedule reboots (which clears out any hanging processes): PsRebooter.
PsManager monitors memory usage over the last 60 minutes and calculates an optimal memory requirement from that. Technically it takes the average of the 8 highest memory usage peaks (discarding the two highest ones) in the last 60 minutes. Single peaks into the memory buffer are acceptable, but regular use of the memory buffer is not advisable as it increases the risk of running out of memory all together. This is why we make an effort to avoid depending on the memory buffer and keep reserved memory on the high end of actual usage.
Dreamhost limits us to making 30 changes per day to the memory size, as a result we have to be a bit careful with changes. To avoid running into this restriction PsManager will never make more than 5 changes in 4 hours. In order to avoid making too many up and down changes, downward changes will occur only if no changes have been made in the previous 120 minutes and the change exceeds 10% of current reserved memory. As a result in any 4 hour period, a minimum of 3 changes are always reserved for upward corrections.
As it is more important to maintain availability of your sites a change will be made upwards if one hour has passed and it exceeds 10% or if the required increase exceeds 20% already after 10 minutes (indicating a possible surge in traffic to your sites).
PsManager will also never use the last available resize in a 4 hour period for a 10% change, but instead reserve it for an emergence upsize of at least 20% as availability of your site is of paramount importance.
Version 0.2.6 also introduces the usage trend in the calculation, if memory usage is going up over an hour it will reserve a bit more than current usage would indicate, and conversely if it is going down it will reserve a bit less.
Versions 0.3.0 and 0.4.0 deal with the continuing changes Dreamhost made to the memory setup of new virtual private servers. We are hoping those changes are now behind us.
As an active user of the tool myself, I am constantly monitoring how it performs and occasionally make tweaks or add features. Suggestions based on your own experience are of course welcome. Some suggestions have led to new features being integrated (for exampe the templated tweets feature).
The PsManager program is released under an open source license (GNU-GPL v3), which means that you can use this script for free and make any modifications that you please. You are in no way dependent on the original author of this program for future maintenance and your right to use it never expires. The source code of the script is easily viewable (the script is written in perl), you can scrutinize it or have it scrutinized to convince yourself that it does what it says it does.
There is a Dreamhost forum thread for this application where you can ask questions. If you are using or want to use PsManager feel free to contact me by E-mail about any problems or suggestions. If you are using PsManager and want to be notified of important new releases, send me an E-mail and ask to be added to the mailing list.
7is7.com | Software | Otto | Travel Stories | Countdown Clock | Firefox | StatEye
New | About | Contact | Connect | Friends | Promotions | Copyright | Advertise
