Difference between revisions of "Dev Maintenance"

From Dreamwidth Notes
Jump to: navigation, search
(Compile the SCSS)
(31 intermediate revisions by 10 users not shown)
Line 1: Line 1:
{{Note|text="We're in the middle of moving over to a git workflow. If you haven't yet moved over to git, see the [http://wiki.dwscoalition.org/wiki/index.php?title=Dev_Maintenance&oldid=10581 old instructions]. If you're looking to move over, first make sure you've finished [[Moving your Dreamwidth installation to use Github]]"}}
 
 
 
== Updating the Dreamwidth code on your Dreamhack ==
 
== Updating the Dreamwidth code on your Dreamhack ==
  
{{Warn|text='''VERY IMPORTANT:''' If you have custom changes that you have not committed or patched out, you want to do that first!  This is very easy. Just use <tt>git stash</tt> to save your work before you start, and recover it after you're done.}}
+
=== Shutting down apache ===
  
 
It is best to shut your Apache instance down before doing the update process, to make sure that everything, especially scripts in cgi-bin/ are reloaded properly:
 
It is best to shut your Apache instance down before doing the update process, to make sure that everything, especially scripts in cgi-bin/ are reloaded properly:
Line 9: Line 7:
 
  stop-apache
 
  stop-apache
  
Okay.  Let's say you've been running your Dreamwidth install and you want to pull down the latest and greatest in fixes.  This is pretty easy.
+
=== Updating dw-free ===
  
You should be making all your changes on branches, leaving the main develop and master branches aloneIf you have any uncommitted changes, first save them:
+
Okay.  Let's say you've been running your Dreamwidth install and you want to pull down the latest and greatest in fixesThis is pretty easy.  First, if you are not somewhere in dw-free, get into that directory:
  
 
  cd $LJHOME
 
  cd $LJHOME
git stash
 
 
You will want to note what branch you are on, too, if you are currently working on one.  It will be starred when you run:
 
 
git branch
 
  
 
Then grab updates from the development branch of Dreamwidth's repository:
 
Then grab updates from the development branch of Dreamwidth's repository:
  
  git checkout develop
+
  git fetch dreamwidth
  git pull dreamwidth
+
  git checkout develop && git pull --ff-only dreamwidth develop:develop
  
You will also want to push the Dreamwidth develop changes to your fork on Github:
+
You will also want to push the Dreamwidth repository changes to your fork on Github:
  
 
  git push origin develop
 
  git push origin develop
  
To restore your saved changes (if you made any), get back to the branch you were on:
+
If you have uncommitted changes, you'll get an error message and will need to use [[Git How To#How_to_stash_your_changes|git stash]].
  
git checkout BRANCH
+
 
git stash pop
+
=== For those using DW non-free ===
  
 
Repeat for dw-nonfree:
 
Repeat for dw-nonfree:
  
 
  cd $LJHOME/ext/dw-nonfree
 
  cd $LJHOME/ext/dw-nonfree
git stash
 
 
   
 
   
# note what branch you are on
+
  git fetch dreamwidth
  git branch
+
  git checkout develop && git pull --ff-only dreamwidth develop:develop
+
  git checkout develop
+
git pull
+
 
  git push origin develop
 
  git push origin develop
 
git checkout BRANCH
 
git stash pop
 
  
Update your database:
+
=== Update your database ===
  
    # order of commands is important
+
Now that your code has been updated, update the database:
    $LJHOME/bin/upgrading/update-db.pl -r -p --innodb
+
 
    $LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb
+
# order of commands is important
    $LJHOME/bin/upgrading/texttool.pl load
+
$LJHOME/bin/upgrading/update-db.pl -r -p --innodb
 +
$LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb
 +
$LJHOME/bin/upgrading/texttool.pl load
 +
 
 +
Remember that different branches might have different text strings you have added, and that you might need to update your database for different branches when you are working on them for those text strings to work.  The same applies for any database changes made by different branches you are working on.
 +
 
 +
=== Compile the SCSS and other static files ===
 +
 
 +
You'll also want to compile any changes to the static files. This will convert SCSS into CSS, as well as recompressing any changed javascript files, into the static build directory:
 +
 
 +
$LJHOME/bin/build-static.sh
 +
 
 +
=== Restart the server ===
  
 
Now you can restart Apache:
 
Now you can restart Apache:
Line 68: Line 66:
 
Instead of using start-apache/stop-apache, which are Dreamhack-specific scripts, use these commands:
 
Instead of using start-apache/stop-apache, which are Dreamhack-specific scripts, use these commands:
  
    sudo /etc/init.d/apache2 start
+
sudo /etc/init.d/apache2 start
    sudo /etc/init.d/apache2 stop  
+
sudo /etc/init.d/apache2 stop  
  
 
You'll also want to update packages on your system at some point. On Ubuntu, this would be done using:
 
You'll also want to update packages on your system at some point. On Ubuntu, this would be done using:
Line 82: Line 80:
 
== Scripting ==
 
== Scripting ==
  
{{Warn|text="These scripts have not been throughly tested yet with the new system; they might not be updated entirely or right."}}
+
{{Warn|text="These scripts have not been thoroughly tested yet with the new system; they might not be updated entirely or right."}}
  
You can, of course, use scripts to make it easier for you to do most of this.   
+
You can, of course, use scripts to make it easier for you to do some of this.   
  
There is an "omnibus" script available at http://dw-dev.dreamwidth.org/94822.html which incorporates all the individual scripts listed here and some other functions, and includes help information.
+
There is an epic "omnibus" script by <dwuser>jeshyr</dwuser> available at http://dw-dev.dreamwidth.org/94822.html which incorporates all the individual scripts listed here and some other useful functions, and includes help information. It'll save you a heap of angst as a beginner!
  
 
Some simpler example scripts are given below.
 
Some simpler example scripts are given below.
Line 95: Line 93:
  
 
<source lang="bash">#!/bin/bash
 
<source lang="bash">#!/bin/bash
oldpwd=$PWD && \
 
cd $LJHOME && \
 
git pull dreamwidth develop && \
 
cd $oldpwd
 
</source>
 
  
Now, when you type 'dwu', this script will update dw-free. If you need to update dw-nonfree as well, then use this code:
+
# make sure we are in the right directory;
 +
cd $LJHOME
  
<source lang="bash">#!/bin/bash
+
git fetch dreamwidth
oldpwd=$PWD && \
+
 
cd $LJHOME && \
+
# pull changes from dreamwidth
git pull && \
+
git checkout develop
cd $LJHOME/ext/dw-nonfree && \
+
git pull --ff-only dreamwidth develop
git pull && \
+
 
cd $oldpwd
+
# push them to Github forks
 +
git push origin develop
 +
 
 +
# restore our old position
 +
git checkout -
 +
 
 +
# check to see if we have a dw-nonfree directory; if we do, update that too
 +
if [ -d "$LJHOME/ext/dw-nonfree" ]; then
 +
  # change to dw-nonfree
 +
  cd $LJHOME/ext/dw-nonfree
 +
 
 +
  git fetch dreamwidth
 +
 
 +
  # pull changes from dw-nonfree
 +
  git checkout develop
 +
  git pull --ff-only dreamwidth develop
 +
 
 +
  # push them to Github forks
 +
  git push origin develop
 +
 
 +
  # restore our old position
 +
  git checkout -
 +
fi
 
</source>
 
</source>
  
 +
Now, when you type 'dwu', this script will update your dw-free and (if you have it) dw-nonfree repositories.
  
 
=== dwdb - Updating the database ===
 
=== dwdb - Updating the database ===
Line 118: Line 135:
  
 
<source lang="bash">#!/bin/bash
 
<source lang="bash">#!/bin/bash
$LJHOME/bin/upgrading/update-db.pl -r -p --innodb && \
+
$LJHOME/bin/upgrading/update-db.pl -r -p --innodb &&
$LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb && \
+
$LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb &&
 
$LJHOME/bin/upgrading/texttool.pl load</source>
 
$LJHOME/bin/upgrading/texttool.pl load</source>
  
 
This will update the database when you type 'dwdb'.
 
This will update the database when you type 'dwdb'.
 +
 +
=== comp - Recompiling the compass files ===
 +
 +
Put this code in a file called "'~/bin/comp'" and make it executable with "'chmod ugo+x ~/bin/comp'":
 +
 +
<source lang="bash">#!/bin/bash
 +
cd "$LJHOME" &&
 +
compass compile &&
 +
cd ext/dw-nonfree &&
 +
compass compile</source>
 +
 +
This will recompile the CSS files.
  
 
==Cleaning up your directories==
 
==Cleaning up your directories==
Line 133: Line 162:
  
 
[[Category: Development]]
 
[[Category: Development]]
 +
[[Category: Dreamhack]]

Revision as of 15:28, 14 March 2017

Updating the Dreamwidth code on your Dreamhack

Shutting down apache

It is best to shut your Apache instance down before doing the update process, to make sure that everything, especially scripts in cgi-bin/ are reloaded properly:

stop-apache

Updating dw-free

Okay. Let's say you've been running your Dreamwidth install and you want to pull down the latest and greatest in fixes. This is pretty easy. First, if you are not somewhere in dw-free, get into that directory:

cd $LJHOME

Then grab updates from the development branch of Dreamwidth's repository:

git fetch dreamwidth
git checkout develop && git pull --ff-only dreamwidth develop:develop

You will also want to push the Dreamwidth repository changes to your fork on Github:

git push origin develop

If you have uncommitted changes, you'll get an error message and will need to use git stash.


For those using DW non-free

Repeat for dw-nonfree:

cd $LJHOME/ext/dw-nonfree

git fetch dreamwidth
git checkout develop && git pull --ff-only dreamwidth develop:develop
git push origin develop

Update your database

Now that your code has been updated, update the database:

# order of commands is important
$LJHOME/bin/upgrading/update-db.pl -r -p --innodb
$LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb
$LJHOME/bin/upgrading/texttool.pl load

Remember that different branches might have different text strings you have added, and that you might need to update your database for different branches when you are working on them for those text strings to work. The same applies for any database changes made by different branches you are working on.

Compile the SCSS and other static files

You'll also want to compile any changes to the static files. This will convert SCSS into CSS, as well as recompressing any changed javascript files, into the static build directory:

$LJHOME/bin/build-static.sh

Restart the server

Now you can restart Apache:

start-apache

Of course, in a production environment, this whole process is not too recommended as you never know what kind of code you're going to get. But for the most part, it's fairly straightforward. (And if you're doing development, this is generally safe.)

Additional Information for Non-Dreamhack Users

This section is only for you if you're running your own installation.

Instead of using start-apache/stop-apache, which are Dreamhack-specific scripts, use these commands:

sudo /etc/init.d/apache2 start
sudo /etc/init.d/apache2 stop 

You'll also want to update packages on your system at some point. On Ubuntu, this would be done using:

apt-get update
apt-get upgrade

Or, if you want an easy command to run on your dw account in one swoop:

sudo apt-get update && sudo apt-get upgrade

Scripting

Warning: "These scripts have not been thoroughly tested yet with the new system; they might not be updated entirely or right."

You can, of course, use scripts to make it easier for you to do some of this.

There is an epic "omnibus" script by [info]jeshyr available at http://dw-dev.dreamwidth.org/94822.html which incorporates all the individual scripts listed here and some other useful functions, and includes help information. It'll save you a heap of angst as a beginner!

Some simpler example scripts are given below.

dwu - Updating the repos

Put this code in a file called ~/bin/dwu and make it executable with chmod ugo+x ~/bin/dwu:

#!/bin/bash
 
# make sure we are in the right directory;
cd $LJHOME
 
git fetch dreamwidth
 
# pull changes from dreamwidth
git checkout develop
git pull --ff-only dreamwidth develop
 
# push them to Github forks
git push origin develop
 
# restore our old position
git checkout -
 
# check to see if we have a dw-nonfree directory; if we do, update that too
if [ -d "$LJHOME/ext/dw-nonfree" ]; then
  # change to dw-nonfree
  cd $LJHOME/ext/dw-nonfree
 
  git fetch dreamwidth
 
  # pull changes from dw-nonfree
  git checkout develop
  git pull --ff-only dreamwidth develop
 
  # push them to Github forks
  git push origin develop
 
  # restore our old position
  git checkout -
fi

Now, when you type 'dwu', this script will update your dw-free and (if you have it) dw-nonfree repositories.

dwdb - Updating the database

Put this code in a file called ~/bin/dwdb and make it executable with chmod ugo+x ~/bin/dwdb:

#!/bin/bash
$LJHOME/bin/upgrading/update-db.pl -r -p --innodb &&
$LJHOME/bin/upgrading/update-db.pl -r --cluster=all --innodb &&
$LJHOME/bin/upgrading/texttool.pl load

This will update the database when you type 'dwdb'.

comp - Recompiling the compass files

Put this code in a file called "'~/bin/comp'" and make it executable with "'chmod ugo+x ~/bin/comp'":

#!/bin/bash
cd "$LJHOME" &&
compass compile &&
cd ext/dw-nonfree &&
compass compile

This will recompile the CSS files.

Cleaning up your directories

If you do any amount of work, you'll find that your directories get cluttered with .orig and .rej files everywhere. This script will clean those up:

git clean -f "*.rej" "*.orig"

You can run it from the command line, or put it in a file called ~/bin/tidy and make it executable with "chmod +x ~/bin/tidy". If you do that, you'll be able to just type "tidy" to clean house.