Skip navigation

Category Archives: unix

Found this gem , with about all you need to know for setting the apache directives on websites.
From the same guy that knows his stuff, this note http://www.askapache.com/security/mirror-using-rsync-ssh.html on using the powerful rsync command

Advertisements

For those that haven’t had the chance to use it, wget is a powerful gnu application that allow to retrieve content non interactively from the web, which makes it ideal to an array of sys-admin tasks. It gets a bit hard to tame so much power and you need to craft the outcome by carefully picking inside the command options. Something practical, like downloading the entire or partial content from inside a folder and its sub-folder, for example, requires a couple of parameters to be passed so not to get confused by the unintuitive way the program goes about retrieving upper-level directories along with the desired stuff.
When you specify the recursive option (-r) to address a site path like http://somesite.com/folder/subfolder/ you would reasonably expect to get every content from that point down the directory tree (much in the fashion that cp -rf would do it, coping all in the hierarchy below) but given the weird behavior of fetching other directories above, the extra option ‘–no-parent’ proves essential.

Along with those two I normally include a couple of extra parameters:

:: ‘-nv; (or ‘–no-verbose‘) that turns off verbose output, without being completely quiet, which means that errors messages and basic information still get printed;

:: ‘-nH‘ (‘or –no-host-directories‘) that disable the generation of host-prefixed directories. By default, invoking wget with ‘-r http://somesite.org’ will create a structure of directories beginning with http://somesite.org

The hypothetical example then will look something like:

wget -r -np -nv -nH  http://www.addressedSite.info/coolJavascriptLib/

The passhtru command in php permits to get the outcome of a unix shell command. If you never knew or heard about it, take a look at it in the manual. It is a fantastic mean to use any shell, perl o unix command from within the php language itself as this allows to output the result of a command in a close fashion to what system() or exec() also do.
In my particular case I ran into a situation hard to debug in which a script was failing only when requested inside another, with either a require or include call. The script failed with a fatal error of memory exhaustion. I didn’t have much time to study and re-code the script to avoid the problem, but it may be worth mentioning the alternative use that helped me to circumvent the failure when I simply used passthru to grab the output of the php script and then print it out inside the function in the way I wanted.

Here’s what I pretty much did:

ob_start();
passthru("php  tree.php --section ".$section." | sed -e '1,8 d'", $notOk);
$tree_outcome = ob_get_contents();
ob_end_clean();

if($notOk==0){
 echo '<div id="siteTree">',$tree_outcome,'</div>';
}

See also that the output is being piped to a sed command only to filter out the first 8 lines that I didn’t use.
As an extra note I had to slightly modify the tree.php script to process the shell argument “section” being passed which now needed to be read inside the $_SERVER[‘argv’] variable.

I don’t know of any drawback or potential problems of this method, it’s a workaround that I was happy to find, so comments from php coders will be very welcome.

This is simply to remember how to change all the DOS “^M” line breaks, from files created in Windows when in need to edit them in a Unix environment.

  • Option 1)
    with Emacs, whith the file open, type:
    M-% C-q C-m RET RET !
  • Option 2)
    use the dos2unix command:
    dos2unix somefile.txt
  • Option 3)
    to apply option 2 inside an emacs buffer:
    C-x h (to select-all)
    C-u M-x shell-command-on-region
    dos2unix RET

Not sure why the default configuration was changed in version 23 for the functions that move cursor around like forward-word, backward-word or back-to-indentation. They highlight the words now. Until I figure it out, (it must be a simply variable to set, any idea?, anyone?) a quick and dirty fix to reset to the previous way I liked, is doing:

;; correcting the way forward-word and similar functions work. 
I don't want them to highlight text while moving cursor.                                                                                                    
(global-set-key (kbd "M-F") 'forward-word)
(global-set-key (kbd "M-B") 'backward-word)
(global-set-key (kbd "M-m") 'back-to-indentation)
(global-set-key (kbd "M-M") 'back-to-indentation)

UPDATE:
Forget about the above rebinding, I just found out what variable we should change to simply return to the Emacs 22 default behavior:

(setq shift-select-mode nil) ; “t” for true, “nil” for false

(Got this at http://xahlee.org/emacs/emacs23_features.html, an impressive site full of valuable information)

The simple solution with awk is doing:

awk ‘{print $NF}’

This is useful when like in the following  we want to list just files skipping directories:

ls -l | awk ‘NR!=1 && !/^d/ {print $NF}

There are other options of course, take a look at this example from a nice collection of bash and unix tips

 

 

To remove empty lines in a region simply do:

M-x flush-lines RET ^$ RET 

or, if blank lines contain some white spaces characters:

M-x flush-lines RET ^\W*$ RET 

Whereas with sed (which I use inside emacs via M-| shell-command-on-region ) it’s simply:

sed -e '/^[	 ]*$/d' 
(this is either a tab or a space, press the TAB key since most versions 
of sed don't recognize the \t character) 

How easy is to forget this type of things!

This is more of quick reminder for myself on the difficult task of escaping quotes and running an awk or sed command inside the bash command line. I solved the issue in my case using the octal representation of a single quote (\47), taken from here, where more options are shown.

Having previously accommodated the list of stale links with their replacements in a two columns format that looked like:

http://addressOld.com http://www.goodWorkingOne.com
http://addressOld2.com http://www.goodWorkingOne2.com
etc…

I used the following:

awk '{print "UPDATE library SET url=\47"$2"\47 WHERE url=\47"$1"\47;"}' 

to correctly generated what I wanted sql-postgres statements to update the links table:

UPDATE links SET url='http://goodWorkingOne.com' WHERE url='http://addressOld.com';

Knowing how to do loops in awk is helpful to address a range of fields. If for example, we want to output only from second, to prior to last field. Say we’ve got :

one:two:three:four:five:six:seven:eight:nine

Fields are here defined by “:” separator, so using:

awk -F ':' '{for(i=2;i<NF;i++) printf "%s:",$i}'




We will get:

two:three:four:five:six:seven:eight:

On how to delete a chunk of text contained in multiple lines: use sed to catch the range (/a/,/b/)

There were some javascript google ads calls from a page I downloaded via wget, doing:

 wget www.someSite.com/somePage.html > toCleanItUp.htlm

After highlighting all the page C-x h I did a M-x shell-command-on-region and used the following:

sed "/<script .*>/,/<\/script>/d" > nowItIsClean.html

If feeling lazy, (or if I don’t have the file opened already in an emacs buffer), there’s the straightforward way of using cat:

cat toCleanItUp.html | sed "/<script .*>/,/<\/script>/d" > nowItIsClean.html

Ah, the beauty of unix tools!