Skip navigation

Category Archives: gnu

place for *nix commands used normally in bash

Now that I’m getting fond of using a heredoc to insert mysql scripts into the bash shell, have put this into my emacs initialization file. The same shortcut (Control+Shift+f11) will conveniently write different parameters depending in which shell I’m sitting in.

UPDATE: seems that I spoke too soon. Asking about the present working directory, formerly “(let ((a (shell-command-to-string “pwd | tr -d ‘\\n'”)))”, didn’t actually work when having opened shells of different servers via Tramp.
The correction below (less elegant) depends on the shell buffer being created/renamed with an identifiable name -which could be the hostname or whatever consistent nomenclature we choose-.
Anyway, until figuring a better way, this does the job of inserting the right parameters into each shell:

(defun my-heredoc-sql-invocation-from-shell ()
"Insert the appropiate parameters to run a heredoc mysql query depending on which shell I'm in"
  (interactive)
  (let ((a (buffer-name (window-buffer (minibuffer-selected-window))))
        (b nil))
    (cond ((string-match "serverA" a) 
           (setq b "mysql -uroot mainDbToHitAt_A--password=`cat /etc/security/mysqlpassword` -t -vv <<\\!"))
          ((string-match "serverB" a) 
           (setq b "mysql -ualpha mainDbToHitAt_B --password=`cat /etc/security/mysqlpassword` -t -vv <<\\!"))
          ((string-match "serverC" a) 
           (setq b "mysql -uroot mainDbToHitAt_C  -t <<\\!")))
    (insert b)))

;; key shortcut to bind it to
(global-set-key (kbd "C-S-<f11>")  'my-heredoc-sql-invocation-from-shell)

Advertisements

I sometimes forget about the exact name of a file I was working a while ago. Luckily the way to state a range of time with find is quite intuitive; say that we want to see the files modified in a lapse of 14 days starting from a week ago:

find . -ctime +7  -ctime -21 # (older than 7 days but newer than three weeks ago)

There are simply 3 options of numeric arguments for time

  1. +n –> More than ‘n’ days ago.
  2. -n –> Less than ‘n’ days ago.
  3. n —> Exactly ‘n’ days ago.

Having discovered the command cpio (wich acts like tar but preserving symbolic links) to recursively put into a single file all the directory structure and files of an entire website, I started to explore its use along with other ways to securely move files between hosts. The command rsync seems like an extraordinary find in that regard.

Here are some neat examples for backing up (or mirroring) sites with ssh I don’t want to miss:

There’s a really easy way to make a remote directory be an exact match of a local directory via an ssh connection. And that’s to use rsync to do the job. It’s smart and only sends enough data to sync the remote dir to the local one. Assuming that you have a local dir named master-copy and the remote system’s repository is in a directory named slave-copy you’d do:

rsync --rsh=ssh --delete -Cav master-copy/* user-name@remote.dom.tld:/path-to/slave-copy

That command will recursively sync everything found in master-copy to the remote. Any files found on the remote that aren’t in master-copy or one of its sub-dirs will be deleted from master-copy. Only changed or new files will be transferred to the remote.

to which another expert adds:

The rsync idea is good — but I for my part always try NOT to install any additional software not essentially required. Today, ssh is “a must” on most systems and you can go with the cpio approach I’ve mentioned.
I must admit the rsync thing is transferring less data if stuff is already on the target side, but this is not the case when running
source=/var/tmp ; target=/var/backup
cd $source ; find . -mtime -1 -type f -print | cpio -ocB | ssh -l username host-b "cd $target ; cpio -icBduml"

daily.

Another suggestion to copy the entire directory tree at /some/path to other.host via SSH is to do:

find /some/path -print | sort | cpio -o -Hnewc | ssh -C other.host "cpio -idvum"

Finally, there’s more to chew with this thorough example on how to mirror a site with rsync http://www.askapache.com/security/mirror-using-rsync-ssh.html

After updating to the newest 1.7.7 version of cygwin (you have to do uname -a to know which one you are running) I found that awk was replaced by gawk, which not only would affect many of my scripts and shell functions but also to me feels harder to type in the command line.

I found that you could simply copy one to another so to have awk as an alias of gawk.
In my setup, all it took was:

cd c:/cygwin/bin
cp gawk awk

This seems a bit idiosincratic, in emacs the way to paste (yank) text in the search minibuffer is doing M-y not C-y (that is Meta-y not Control-y, as is the default way for all other matters)
It’s funny that I didn’t realize something so basic until now. I don’t know what the reason might be, but this is still the way at least in emacs 23.1.91
(R. Stallman answered the same question in 2005 here at http://www.mail-archive.com/emacs-pretest-bug@gnu.org/msg03693.html)

UPDATE: maybe worth pointing that when editing a search (M-E) you need to use the canonical C-y to yank instead of M-y as it is the case in the search

For those that haven’t had the chance to use it, wget is a powerful gnu application that allow to retrieve content non interactively from the web, which makes it ideal to an array of sys-admin tasks. It gets a bit hard to tame so much power and you need to craft the outcome by carefully picking inside the command options. Something practical, like downloading the entire or partial content from inside a folder and its sub-folder, for example, requires a couple of parameters to be passed so not to get confused by the unintuitive way the program goes about retrieving upper-level directories along with the desired stuff.
When you specify the recursive option (-r) to address a site path like http://somesite.com/folder/subfolder/ you would reasonably expect to get every content from that point down the directory tree (much in the fashion that cp -rf would do it, coping all in the hierarchy below) but given the weird behavior of fetching other directories above, the extra option ‘–no-parent’ proves essential.

Along with those two I normally include a couple of extra parameters:

:: ‘-nv; (or ‘–no-verbose‘) that turns off verbose output, without being completely quiet, which means that errors messages and basic information still get printed;

:: ‘-nH‘ (‘or –no-host-directories‘) that disable the generation of host-prefixed directories. By default, invoking wget with ‘-r http://somesite.org&#8217; will create a structure of directories beginning with http://somesite.org

The hypothetical example then will look something like:

wget -r -np -nv -nH  http://www.addressedSite.info/coolJavascriptLib/

Hand editing web stuff (outside of wysiwyg editors, that is) a lot of times requires dealing with messy html markup. Even though tags are meant to be parsed by browsers, and there are even performance benefits for serving ugly obfuscated code without white spaces, the human readability of the markup needs some tidiness in the formatting of web pages for which the indentation of tags makes us able to understand what goes on inside it.

Emacs has a large set of commands to internally handle code indentation, but never occurred to me until recently that they could be helpful to simply re-align the clutter of html tags that you can see from the output of a source view.

I went on defining something basic like this little elisp function to quickly help with the general re-alignment of code.

(defun my-tidy ()
  "Automatically re-indents code"
  (interactive)
  (execute-kbd-macro
   (read-kbd-macro "C-u -999 M-x indent-region RET C-x C-x M-x indent-region")))

(global-set-key (kbd "C-M-*") 'my-tidy)

Now that was fine but still insufficient. Say, what about the embedded css or javascript that normally goes along inside a web page?
I didn’t like the way emacs (in its default mode, of course) gets css code re-aligned, plus I actually do have specific requirements which demand the ability to reformat the css style sheets declaration and rules when editing a web document. As I use firefox with its web-developer plug-in, in order to save screen real state I leave the narrowest possible window at my left, therefore it is inconvenient to have css stuff deeply indented to the right (wanted just one space from left margin), additionally, rules needed to be broken up nicely as well (avoiding long chains like #div a.rule1, #div b.rule2, #div c.rule 3 for example).

This screen-shot might show better why the formatting of css code matters so much in my work setup.

my css editing style

So playing a bit more with the idea, I went on crafting some good old sed and awk one-liners for reformatting all what could be found embedded inside <style> tags. The formidable shell-command-on-region in emacs allows such things. You will note that the regexp ain’t look pretty,  it’s sort of long (I might make another post explaining how it breaks up), and also has the dreaded leaning toothpick syndrome! cause emacs lisp needs characters to be double escaped.

In short, all what I wanted is wrapped up below exactly as it now goes inside my emacs init :

(defun select-css-code ()
  "Select region contained by <style></style> tags.
   Simply highliths what is between those tags for embedded css content"
  (interactive)
  (save-excursion)
  (let(p1 p2)
    (goto-char (point-min))
    (search-forward "<style ")
    (backward-char 7)
    (setq p1 (point))
    (search-forward "</style>")
    (setq p2(point))
    (goto-char p1)
    (push-mark p2)
    (setq mark-active t)))

(defun select-javascript-code ()
  "Select region contained by <script></script> tags.
   Simply highliths what is between those tags for embedded javascript content"
  (interactive)
  (save-excursion)
  (let(p1 p2)
    (goto-char (point-min))
    (search-forward "<script ")
    (backward-char 8)
    (setq p1 (point))
    (search-forward "</script>")
    (setq p2(point))
    (goto-char p1)
    (push-mark p2)
    (setq mark-active t)))

(defun re-indent-web-page-code ()
  "Re-indents html code including its embedded javascript and css.
The css code gets indented diferently (through some awk and sed one-liners)
to ease the editing of styles with Firefox using a window of its Web-Developer plug-in."
  (interactive)
  (progn
    (mark-whole-buffer)
    (indent-rigidly (region-beginning)(region-end) -999)
    (indent-region (region-beginning) (region-end))
    (select-javascript-code)
    (javascript-mode)
    (indent-rigidly (region-beginning)(region-end) -999)
    (indent-region (region-beginning) (region-end))
    (html-mode)
    (select-css-code)
       (setq command  "awk  '/{/ {gsub(/,/,\",\\n\")} {print }' | sed -ne '1h;1!H;${;g;s#{\\([^\\n]\\)#{\\n\\1#g;p}' | sed -ne '1h;1!H;${;g;s#;}#;\\n}#g;p}' | sed -ne '1h;1!H;${;g;s#;\\([^\\n]\\)#;\\n\\1#g;p}' |  sed -e 's#^[ \\t]*\\(.*\\)$#\\1#g' |  awk  '!/{|}|^ *#/&&!/^\\// {$0 = \" \"$0} {print }'  | awk NF |  sed -e 's#\}$#}\\n#g' | sed -ne '1h;1!H;${;g;s#,\\n#,#g;p}' | awk  '/{/ {gsub(/,/,\",\\n\")} {print }' | sed -e 's#^[ \\t]*\\(.*\\),$#\\1,#g'" )
      (shell-command-on-region (mark)(point) command t t)))

UPDATE: note that instead of a macro call

    (execute-kbd-macro
     (read-kbd-macro "M-x indent-region"))

I’m using this straight forward lisp expression

    (indent-region (region-beginning) (region-end))

TODO:
The function is almost there, I still need to address a couple of things:
What if we have many [style or javascript] sections intermingled in our html?
One way to address that would be to successively grab the content, send it to other place using the acummulating-text function of emacs like ‘append-to-buffer ( http://www.gnu.org/software/emacs/manual/html_node/emacs/Accumulating-Text.html), then, we can switch to that second buffer, treat the code there and simply get it back to the original document.
Also noted that the css code doesn’t get indented if the javascript tags aren’t found, so I’ll revise the logic to allow it regardless of whether javascript code exist or not.

My quick guide for checkboxes
I like them but can easily forget about the simplest detail of their creation and use:

  • to create checkboxes use “[0/0]” or “[0%]” ( both can also be together in any order)
  • doing “C-#” recalculates the counts (updates the checkbox statistics)
  • add a “- [ ]” (dash space left-braket space right-bracket) at the beginning to create a checkbox item
  • new items can be created from there doing “S-M RET” (before, or after depending on whether cursor was at the beginning or end of item)
  • doing “C-c C-c” you can toggle check marck on/off
  • with “C-x-b” toggle checkbox at point, which according to the online manual works like the following:
    • If there is an active region, toggle the first checkbox in the region and set all remaining boxes to the same status as the first. If you want to toggle all boxes in the region independently, use a prefix argument.
    • If the cursor is in a headline, toggle checkboxes in the region between this headline and the next (so not the entire subtree).
    • If there is no active region, just toggle the checkbox at point.
    • ( this is found at http://www.gnu.org/software/emacs/manual/html_node/org/Checkboxes.html )
** My quick guide for checkboxes [2/3] [66%]
This is a small sample
- [ ] item 1
- [X] item 2
- [X] item 3 which containes 4 sub items
    * sub item 3.1
    * sub item 3.2
    * sub item 3.3
    * ( see more at http://www.gnu.org/software/emacs/manual/html_node/org/Checkboxes.html )

The passhtru command in php permits to get the outcome of a unix shell command. If you never knew or heard about it, take a look at it in the manual. It is a fantastic mean to use any shell, perl o unix command from within the php language itself as this allows to output the result of a command in a close fashion to what system() or exec() also do.
In my particular case I ran into a situation hard to debug in which a script was failing only when requested inside another, with either a require or include call. The script failed with a fatal error of memory exhaustion. I didn’t have much time to study and re-code the script to avoid the problem, but it may be worth mentioning the alternative use that helped me to circumvent the failure when I simply used passthru to grab the output of the php script and then print it out inside the function in the way I wanted.

Here’s what I pretty much did:

ob_start();
passthru("php  tree.php --section ".$section." | sed -e '1,8 d'", $notOk);
$tree_outcome = ob_get_contents();
ob_end_clean();

if($notOk==0){
 echo '<div id="siteTree">',$tree_outcome,'</div>';
}

See also that the output is being piped to a sed command only to filter out the first 8 lines that I didn’t use.
As an extra note I had to slightly modify the tree.php script to process the shell argument “section” being passed which now needed to be read inside the $_SERVER[‘argv’] variable.

I don’t know of any drawback or potential problems of this method, it’s a workaround that I was happy to find, so comments from php coders will be very welcome.

This is simply to remember how to change all the DOS “^M” line breaks, from files created in Windows when in need to edit them in a Unix environment.

  • Option 1)
    with Emacs, whith the file open, type:
    M-% C-q C-m RET RET !
  • Option 2)
    use the dos2unix command:
    dos2unix somefile.txt
  • Option 3)
    to apply option 2 inside an emacs buffer:
    C-x h (to select-all)
    C-u M-x shell-command-on-region
    dos2unix RET