[Trisquel-users] Grep consumes all my RAM and swap ding a big job

amenex at amenex.com amenex at amenex.com
Sun Jul 28 13:58:15 CEST 2019


Magic Banana asked:

 >> Here is what I actually proposed (where filenames are appended *before*  
the join):
 >>  
https://trisquel.info/forum/grep-consumes-all-my-ram-and-swap-ding-big-job#comment-142474

When I collapsed the script into a one-liner, it would start ... but nothing  
ensued for about ten minutes.
At the point of desperation, it dawned on me what you proposed, so I put it  
in more readable terms:

 > time awk '{print FILENAME,"\t",$0}' 01.txt 02.txt 03.txt 04.txt 05.txt  
06.txt 07.txt 08.txt 09.txt 10.txt 11.txt 12.txt 13.txt 14.txt 15.txt 16.txt  
17.txt 18.txt 19.txt 20.txt 21.txt 22.txt 23.txt 24.txt 25.txt 26.txt 27.txt  
28.txt 29.txt 30.txt 31.txt 32.txt 33.txt 34.txt 35.txt 36.txt 37.txt 38.txt  
39.txt 40.txt 41.txt 42.txt 43.txt 44.txt >  
Backup/ProcessedVisitorLists/FILENAME.txt

This script is expandable for the foreseeable future, because rather few  
webmasters publish their Webalizer data online.
I wrote it for the joined files, but it's as easily applied to the pre-joined  
files as well. I've since changed my naming
protocol to make the status of the files more clear.

 >> Can't you write *.txt?!
FILENAME has specific meaning in awk, so I was sure that I would be getting  
that for which I was asking. Besides: it works.


More information about the Trisquel-users mailing list