Forum Posts Feed
Artisanal software sale
< Next Topic | Back to topic list | Previous Topic >
Posted by Andy Brice
Jun 26, 2021 at 12:49 PM
>I just saw the video for Easy Data Transform - impressive!
Thanks. The video is of a rather old version. I need to re-do it.
>Our local property tax district is out of control. I asked for a copy of
>the tax roll and got a text file containing a single line of 336 million
>characters. Deliberately, I’m sure, no explanation of the data format
>was given.
That is pretty inexcusable.
“All professions are conspiracies against the laity” (George Bernard-Shaw)
>
>Some research in their budget revealed a software vendor which led to
>documentation of the file format. I’m sure they intended me to not make
>sense out of what they sent. Python is my friend, and I wrote a class
>that parses the data, which is about five data structures intermingled.
>
>Curiously, there are totals records in the file which are wildly wrong.
>I’m very curious what else I can find.
>
>Rather than spend a lot of time coding, I’m going to see what Easy Data
>Transform will do, either with the raw fixed length records or with
>excerpts I can write from the file in CSV or Excel format.
You can often combine transforms together to do things that aren’t immediately obvious. Just ask on https://forum.easydatatransform.com/ if you get stuck on anything.
Andy Brice
Posted by Amontillado
Jun 26, 2021 at 04:57 PM
Apologies if this is a duplicate post. My browser burped. I don’t see my post. I resurrected the below from browser history.
Andy,
I bought Easy Data Plan. Currently, I have only one use for it. I could easily complete my data examination in the trial period, but that would leave me vulnerable in the future.
After all, Marshall Dillon doesn’t settle his differences at high noon with a six-gun out on a test drive.
Batman doesn’t rent his cape.
Easy Data Transform easily handled 350 meg of tax roll data. The only thing I had to do was add a line feed after each fixed length record and write different embedded record types to different files. Pfft, mere seconds of work in Python.
I had hundreds of columns to rename. There were bunches of repeated columns non-nerds would need to retype repeatedly. Keyboard Maestro solved that, but it would be a nice feature to be able to highlight a range of column names in the “rename” transform, copy, and paste either into additional columns, or to duplicate in a different transform.
There were also cases where I skipped a column in the rename transform. That would have meant retyping columns from the point of error forward, over a hundred on my first such goof. Again, KM did the trick with a macro to start at the bottom and jog each name down by one.
Perhaps the coolest cure would be able to load column names from a single-line CSV file. Or, better yet, slurp off the top line of a CSV to rename columns.
Anyway, I hate to gush, but Easy Data Transform is really cool. IBM InfoSphere would do the trick, too, but I don’t know how I could hide the $2,500/month subscription from my wife. Altair has a nice solution, too, but at $2,000/year it would still leave a mark on my wife’s pesky spreadsheets.
Given the price of other solutions, I owe you an apology for buying Easy Data Transform at the SummerFest pricing. Really crass of me, but, see above, my wife’s spreadsheets are a force.
And, snickering with the panache of Snidely Whiplash twirling his mustache, I now have data science tools for analyzing her spreadsheets and correlating her Amazon purchases!
Er, wish me luck on that one.
Posted by Andy Brice
Jun 27, 2021 at 08:48 AM
>Easy Data Transform easily handled 350 meg of tax roll data. The only
>thing I had to do was add a line feed after each fixed length record and
>write different embedded record types to different files. Pfft, mere
>seconds of work in Python.
You might have been able to do that with the ‘split rows’ transform. Hard to say without seeing your data.
>
>I had hundreds of columns to rename. There were bunches of repeated
>columns non-nerds would need to retype repeatedly. Keyboard Maestro
>solved that, but it would be a nice feature to be able to highlight a
>range of column names in the “rename” transform, copy, and paste either
>into additional columns, or to duplicate in a different transform.
>
>There were also cases where I skipped a column in the rename transform.
>That would have meant retyping columns from the point of error forward,
>over a hundred on my first such goof. Again, KM did the trick with a
>macro to start at the bottom and jog each name down by one.
>
>Perhaps the coolest cure would be able to load column names from a
>single-line CSV file. Or, better yet, slurp off the top line of a CSV to
>rename columns.
You can substite a header row from another file. Just input it and ‘stack’ it on top of your data (align by column number).
I have also thought about providing find and replace as part of the ‘rename cols’ transform.
The ‘split rows’ might have avoided having so many columns.
>
>Anyway, I hate to gush, but Easy Data Transform is really cool. IBM
>InfoSphere would do the trick, too, but I don’t know how I could hide
>the $2,500/month subscription from my wife. Altair has a nice solution,
>too, but at $2,000/year it would still leave a mark on my wife’s pesky
>spreadsheets.
>
>Given the price of other solutions, I owe you an apology for buying Easy
>Data Transform at the SummerFest pricing. Really crass of me, but, see
>above, my wife’s spreadsheets are a force.
At least with the Summerfest promotion, I’m not paying any commission, just an up-front contribution to the publicity. Thanks Mark Bernstein of Tinderbox (the organizer)!
>
>And, snickering with the panache of Snidely Whiplash twirling his
>mustache, I now have data science tools for analyzing her spreadsheets
>and correlating her Amazon purchases!
>
>Er, wish me luck on that one.
Software can only do so much. ;0)
We are a bit off-topic. But feel free to email me Easy Data Transform related questions at the support email (https://www.easydatatransform.com/support.html) or post on https://forum.easydatatransform.com/.
Posted by Drewster
Jun 27, 2021 at 02:22 PM
I wish I had a use for Easy Data Transform. I watched the demo video and was amazed by the power and ease of use.
I am a happy user of HyperPlan and can vouch for Andy’s responsiveness as a developer.
Posted by Andy Brice
Jun 27, 2021 at 05:19 PM
Thanks! I really most update that video…