Topic: Apache, Mongrel, Subversion, Rails... can they dance?

I know they can. HOW is another thing.

After 8 weeks of anxious waiting I got a SliceHost slice. I installed Ubuntu LTS (Dapper), Apache 2, mySQL, PHP5, Ruby, Ruby Gems, Rails, Subversion...

SliceHost has a lot of good documentation... I've spent the last few days reading it but, as with most documentation its only good when it works. I'm starting this thread in hopes someone(s) can help me tie a few shards of information together where my mind missing the train.

Also, I think I've ranted about it before around here, but I am now on my 7th HONEST ATTEMPT to understand Subversion and its NOT clicking. At some dark moments I find it hard to believe that I'm a mildly successful web professional.

The plan is to host multiple web sites (defined for the sake of this discussion as a domain and the files associated with it). Some will need subversion, some won't. Some will be RonR, some PHP.

I have no problem configuring a sites-available file in apache. I am having trouble putting a project in Subversion and I'm not even sure how the directory structure should be despite the oh-so-helpful Subversion documentation. sad I initially wanted it in a /var/ folder rather than a user folder because multiple users would access it... so I ran the svnadmin command and make my folder, did the svn import on what I had of a holding page... that was all fine.

Then I realized that not having the files in a user folder made uploading files kind of a mess, so I decided to abandon that tactic and rather do my folders under the user (/home/username/... ). Did the svnadmin thing again, tried the import thing again... it acted like it worked.... showed all the files it was checking in but... they're nowhere to be found. Certainly not where I specified on the import (and its a no-brainer as they are all absolute paths).

But I'm not here just to bemoan Subversion. Another guy and I are setting up for a Rails project... that means two people accessing code, we want the rollback power for when we mess things up (hence Subversion). So I want the power of MCV, of development, test, and production... all to make the ideal development environment.

So I'm trying to see the three dimensions... files, databases, versions, development...

Do you treat your development site like its own web site entirely? Like a set of working files? Or a branch? Do I give it it's own sub-domain and secure it somehow? Should I have another site & sub-domain for test and production each? Or is there a way I'm missing?

And what makes one go to another? What puts our dev changes into test, and test into prod? Subversion? How does Rails know or does it need to?

Do I create a Rails project in a Subversion folder or do I have to build it beforehand? And in rails what folder is Apache going to want to put up? Nobodies documentation is answering this for me!!!

(I'm feeling the aggrivation of three days of cutting my teeth on all this stuff)

My AIM is Pier2Degin if anyone is daring.

Thanks

"Bear 270, young man. Bear two, seven, zero, over." - Musings of a flight simulator guru, me.

Re: Apache, Mongrel, Subversion, Rails... can they dance?

I don't have a lot of time, so I'll answer one question.

And what makes one go to another? What puts our dev changes into test, and test into prod? Subversion? How does Rails know or does it need to?

Rails, by default, has the three Environments (development, test, and production).  In general, these three environments share all the same files, but different databases (the test environment, for example, NEEDS it's own database, as it nukes the db everytime you run a test (to make sure you've got fresh data) and rebuild it from the test runners you have set up).  To specify the environment, you'd normally set it via the RAILS_ENV environment variable which would be handled by your server. 

To be honest, as you have hosting you can control, I would recommend either lighttpd or Monger to act as your Rails server and just proxy the connect with Apache, as Apache + fast_cgi + Rails = pain in the butt + memory hog.  Then what you can do is have requests either to a subdomain or different port be mapped to different environments.

If you're still having trouble with SVN, I'll look through my book (yes, I would qualify as geeky for having the official SVN book big_smile) and see if I can't write something up (or find an easy-to-use tut on the web somewhere).

Last edited by pnomolos (2007-11-18 23:21:20)

"Take up your cross before your crown" :: http://no-spec.com

Re: Apache, Mongrel, Subversion, Rails... can they dance?

Hey, I'll certainly take one answer!

A few things you said I didn't fully grasp: "...you'd normally set it via the RAILS_ENV environment variable which would be handled by your server. "

Is RAILS_ENV a config setting in Rails? What do you mean "handled by [my] server"? BTW I have the Agile Dev w Rails book but it has 1 page on development environment.


Besides digging around the SVN docs I've tried following two tutorials:

One from Slicehost (Not Rails Specific): http://articles.slicehost.com/2007/9/5/ … subversion
Another from somebody: http://maniacalrage.net/past/2006/4/12/ … _create_a/

Actually Slicehost has no less than five SVN articles but as I said, it works when everything goes right. I get lost on both tutorials at exactly the same point. When you check out to a working directory, is that working directory entirely outside the SVN tree? Or does it have to be inside? But the latter doesn't make sense, because what if your working directory is local while the Slice as we know is remote?

Correct me if I'm off, but would it be a viable option to have a repository in one place on the server, and an easy to SFTP access working directory somewhere else on the server? (Part of the reason I'm thinking this is both me and my programmer guy are on Windows, although we have done the Ruby, WAMP, and MySQL local dev thing) Otherwise how does SVN check files to a local machine from the remote server?

Sorry to ask 10 questions when you're trying to answer one. I'd really like to learn this and know what I'm doing. I've been dependent on managed hosting WAY too long!

"Bear 270, young man. Bear two, seven, zero, over." - Musings of a flight simulator guru, me.

Re: Apache, Mongrel, Subversion, Rails... can they dance?

How you setup your slice depends on quite a number of factors, including: 1) who will be accessing it (just you and your development team, or the customer), 2) what kinds of hosting you are providing (just Rails, Rails + PHP, etc) and 3) the size of the slice you selected. Of course, don't exclude personal taste and experience with specific technologies that weigh into the mix...

My approach is to have the customer purchase the slice themselves, then host their own svn repository plus DB, Rails, etc. on their own slice. They can then grow it or add more as desired, plus they own the code. If you are going to host the code and/or Rails processes plus manage the slices yourself, then the approach you select may be different.

Here are some more details that will (hopefully) clarify things:

For SVN, it can be accessed either a) directly on the machine via the filesystem, b) using an ssh connection (svn+ssh://), or c) using http via Apache + Webdav + svn modules. Which one you use depends on the answer to question 1 above (who) - most often it is either using ssh or http. Where to put the repository also depends on who will be accessing it - a home directory with permissions to a specific group of users, or to anyone in your company that has a user account. If you use Apache to frontend svn, you can use the http authentication to create groups and users that each have permissions to access specific svn repositories. Coupled with a custom Trac installation that I do, I can host multiple projects with svn + Trac, plus have ticket tracking and a wiki for tracking knowledge using a single slice (I do this for my company slice, but not for customer-specific slices since they are more simplified needs).

For SVN projects, I typically create a new svn repos for each client, and within that have a trunk, branches, and tags directory at the root. From trunk, I create webapp, where I store my Rails app (some people just shove it inside trunk), and a docs for storing MS Office docs, diagrams, etc. during the design phase. I used branches for parallel development or "what if" scenarios, whereas I use tags for marking releases to production that I may want to revert to in the future.

For Rails, 3 environments are provided by default - development, production, and test. Each one is configured for specific reasons (development doesn't blow up if a mail server isn't found, while production does). You can understand more about what they do by reading the various config/environments/*.rb files. Typically, development runs on your local box, as does test (for your unit/functional/integration tests) and production is used in a production environment.

For Rails deployment, I use nginx for the frontend load balancing to 2+ mongrel processes. The mongrel configuration file allows you to specify the environment to run (typically production), which in turns points to the correct DB servers in databases.yml. Each mongrel process will need from 20-50MB of RAM each, so you'll want a min of a 512MB slice for anything beyond the smallest of hosting needs. nginx is great, as it can be pointed to your images/stylesheets/etc and serve them to the browser like Apache, without all of the overhead of Apache. Of course, if you are using Apache for svn http, then you may just want to use Apache plus a module to balance the incoming requests targeted at the mongrel processes.

For databases, since I have each customer provide their own slice, I setup their DB (just like their svn and Rails processes) on their slice. If you are going to host multiple customers, you can either a) setup multiple databases, named differently with a customer prefix (customer_production) or b) setup multiple db processes on different ports. Option a requires less memory but you must be careful not to allow logins for the database to have access to another customer's data. Option b requires more setup and memory, but keeps things completely separate.

For deployment, I use Capistrano scripts to automate the setup and deployment of my code to the slice, along with restarting the processes once the code is deployed. I also use Marshmallow to send an automated message to my Campfire chat forum to let others know when a new version has been deployed.

Hopefully this will help get you going. I'm actually considering a service for my consulting company where I can help get projects going using Slicehost. If you are interested, drop me a note using my contact form on my website: http://www.bluejazzconsulting.com

Re: Apache, Mongrel, Subversion, Rails... can they dance?

Leovenous wrote:

A few things you said I didn't fully grasp: "...you'd normally set it via the RAILS_ENV environment variable which would be handled by your server. "

Is RAILS_ENV a config setting in Rails? What do you mean "handled by [my] server"? BTW I have the Agile Dev w Rails book but it has 1 page on development environment.

RAILS_ENV is an environment variable that you could set in Aoache/Mongrel/Lighttpd/etc, but you'd have to check documentation.  For example, though, in Apache you could do the following in your .htaccess/httpd.conf file:

SetEnv RAILS_ENV development

This would set the RAILS_ENV variable.  You could have it in an .htacess file in your main Rails project and just change it to set what mode the site is in.

"Take up your cross before your crown" :: http://no-spec.com

Re: Apache, Mongrel, Subversion, Rails... can they dance?

Here's how my workflow goes:

1. Create the project:

rails my_project
cd my_project

2. Create a new subversion repository for the project. I use a separate subversion repository for each project. This helps to keep things separate. How you create repositories depends on your host.

3. Import the project into the SVN repository:

svn import . http://my_domain.com/svn/my_project/trunk

This command imports the current directory (the period means "this directory" - make sure you're in the "my_project" directory) into the URL specified. The URL would be the URL for the repository that you just created. I usually import the code into the "trunk" directory under the repository. This is just a Subversion convention.

4. Check out the project:

cd ..
mv my_project my_project.old
svn checkout http://my_domain.com/svn/my_project/trunk my_project

Move the original files aside to make room for the checked out code. Then check out the code from the actual subversion repository. If you skip this step, you'll be working on files that aren't part of the repository, and you won't be able to commit your changes. After you checkout the files from the repository, any changes that you make can be committed, and your repository will stay in sync.

5. Make Changes. Commit. Repeat.

svn commit

After you type that command, an editor window will come up and ask for a description of the changes you made. It's a good idea to be as descriptive as possible. This will help in the future when you're trying to track down a bug or put together release notes.

6. Update

svn update

Before you start working, it's a good idea to run svn update to make sure you have the most recent working copy of the code. If your coding partner has committed any changes, this will update your copy.

7. Test Locally

Here's where RAILS_ENV comes in. As others mentioned, you have three environments; development, test, and production. In general:
* "test" is reserved for tests
* "development" is for when you run "./script/server" on your local development machine
* "production" is for when you put your app on the server

You need one database for each of the environments. If you want, on your local development box you can use the same database for both production and development, but you MUST have a separate database for test. This is because the test database gets completely nuked each and every time you run your tests.

I like to develop and test with SQlite3 as a database. SQLite is nice because it's file-based, so the database is really portable. You can zip the database, email it, upload it, include it in the repository, whatever. Very convenient. Also, in some cases you can use a memory-based SQLite database, which can make for faster tests. For production I use PostgreSQL.

8. Deploy

I *highly* recommend Capistrano (http://www.capify.org/) for deployment. Highly.

As a general rule, I only run production mode on the server, and never development mode. This is because development mode loads the entire Rails stack for each and every request. Development mode will eat your server for lunch and have your sysadmins breathing down your neck. Depending on your host, you may also get stuck with CPU and memory overages.

So, to summarize...

RAILS_ENV:
* development mode is for your local development box
* test mode is for testing
* production mode is for the server

Subversion:
* One subversion repository per project
* Import first, then check out the project with svn checkout
* svn commit: Commit your local changes
* svn update: Update your local copy of the repository with any changes that have been made by others
* Deploy with Capistrano

Ryan Heneise  |  Art of Mission  |  Now with extra-strong Donor Tools mojo

Re: Apache, Mongrel, Subversion, Rails... can they dance?

This is good... lots of feedback. I'm reading and re-reading these.

@jhiggenbotham: Sounds smart to offer consulting on this. There seems to be so little real standardization on development environments and server configs.

@ryenski: Hmmm... production on the server only... hmmm. Perhaps that is why I found one place where a guy said to find any old box laying around and set it up as a testing server. Maybe this is another crazy question, but doesn't committing it to the production repository make it live? Isn't it a good idea to see it running on your server (or Slice) before the world does? In my case I'd like to see what my programmer guy does before it goes live. Perhaps there can be two production folders, one for almost-production testing and another actually live? Maybe I'm revealing more of my ignorance of SVN.

Time to mess around some more.

"Bear 270, young man. Bear two, seven, zero, over." - Musings of a flight simulator guru, me.

Re: Apache, Mongrel, Subversion, Rails... can they dance?

Leovenous wrote:

Isn't it a good idea to see it running on your server (or Slice) before the world does? In my case I'd like to see what my programmer guy does before it goes live.

Yes, this is why many people set up a fourth environment called staging. Staging is basically an identical copy of the production environment where you can test deployment.

Also, you can run production mode on your local computer. In your project folder, type "./script/server RAILS_ENV=production". This will start up the local instance of your Rails app in production mode.

Leovenous wrote:

Maybe this is another crazy question, but doesn't committing it to the production repository make it live?

No, Subversion and Rails actually have nothing to do with each other. Committing an update to the repository simply commits any changes that you've made to the repo. Here's an example:

1. You sit down to write some code. You use "svn update" to update to the latest revision: Revision 100.
2. You commit your changes. Revision is now 101.
3. You deploy Revision 101 to your production server.
4. You continue to work. You commit revisions 101, 102, 103, etc...
5. The server is still running Revision 101. The server will run R101 until you deploy a new version to the server.

A good book (if slightly technical) about Subversion is Pragmatic Version Control

Ryan Heneise  |  Art of Mission  |  Now with extra-strong Donor Tools mojo