Tuesday, November 26, 2013

Linux find certain files containing text and replace that text

Here we go!  This was a fun command to type out.

  • Find files in the current and sub-directories matching the name *Spec*
  • Files must contain the "search_text"
  • Replace search_text with repalce_text

find . -name "*Spec*" -exec grep -l "search_text" {} \; | xargs perl -pi -e "s/search_text/replace_text/"

Danger! Be careful with this command.  Don't blame me if you mess up all your files!

Wednesday, November 6, 2013

Download data from Angular ngGrid as CSV

I have an Angular ngGrid on my page.  I want the user to be able to download a spreadsheet copy of the data.  Ideally I want the filtered copy of the data.

First - ngGrid CSV Export Plugin

First I tried the ngGrid plugin, CSV Export.  It worked fine until I threw 10,000 rows at it.  Then it would literally add time to my page load in the magnitude of 10 seconds.  Also, once I got over about 6,000 rows, clicking the CSV Export button would cause my page to crash.  (Using Chrome 26)

This technique relies on data urls.  Read more about those here:

This is no offense to the creator(s) of the CSV Export Plugin, just my experience as of this writing.

It's possible that a data URI will still work as per this example: http://stackoverflow.com/a/15690552/1804678

Second - download hyperlink

The second option I tried was to create a hyperlink that would download the data by calling a service (provided by node.js).  This is a fairly simple option, but it requires a duplicate service call to the one that loaded my grid in the first place; requires that I edit the service to support returning CSV; and will not give me the filtered rows of the grid.  Anywho, I added this to my html:

<a ng-href='/history/{{qs}}'>download spreadsheet</a>

And in my route (node.js):

exports.history = function(req, res) {
'Content-Disposition' : 'attachment; filename="a.csv"',
'Content-Type' : 'text/csv'

res.send("\"a\",\"b\"\n1,2\n3,4"); // Hard coded example.

This works, but I'm still not really satisfied.

Third - mini form submit

This is a cool idea I found based on a stack overflow post.  Basically, form submit JSON from your $scope to a service.  This would be similar to the second option, but it uses a form in the html instead of a hyperlink. Also it is a post, so that you are actually posting the JSON data to the back end, convert the data there from json to CSV (using json2csv).  Then return the data as an "attachment".  You can't use the node/express "attachment" function since that takes a file, but you can mimic the attachment function by setting the headers yourself (also shown in the second option above).

This option works really well.

Here's the HTML:

<form action='/json2csv' method='post'>
<input type="hidden" name="json" value="{{mydata}}" />
<input type='submit' value="3. Download Spreadsheet"/>

And here's the server side node code:

exports.json2csv = function(req, res) {
var jsonObj = JSON.parse(req.body.json);
var fields = Object.keys(jsonObj[0]);
"data" : jsonObj,
"fields" : fields
}, function(err, csv) {
if (err)
'Content-Disposition' : 'attachment; filename="a.csv"',
'Content-Type' : 'text/csv'