Word frequency counters and some concordancers (a prerequisite for smart text expander = autocompleter usage)

Posted by 22111
Oct 31, 2014 at 01:03 PM

This is just for adding another aspect to this monologist thread: Word lists ain’t everything, even with context (concordance). (And within parentheses: Some people really try very hard, and deliver lots of value to their customers, for quite very modest sums of money over the years, Zoot being an outstanding example of this way of treating your customers as a real friend. And other developers seem to work the easy way, offerings lots of lists and such, with lots of code recycling, but also with very frequent paid “upgrades” to what finally does not bring anything really new AND exceptional, and where, paid upgrade after paid upgrade, the really needed things continue to be missing. Anyway, my comment on SmartEdit for MS Word over at bits will certainly put into perspective some of the alleged utility of easy-to-code list fabrication:)

http://www.bitsdujour.com/software/smartedit-for-word#comments94403

Some remarks.

SmartEdit (not for MS Word) had been on offer some time ago; there also was a cut-down free edition available, which does not seem to be available anymore.

I quickly disenchated from my trial; in fact so fast that I even had time to decide I finally did not want to buy even at the very low price for the full version, and afterwards, I never used the free version either - but this is all me, you mileage might grossly vary.

It’s understood SmartEdit does not offer spelling correction, so you need some dedicated tool for that, or you need that functionality to be integrated within your main program; fortunately this is the case with MS Word.

At the time, the developer spoke of his intention to cut his program into two: One for helping editing fiction and such, and one for textbooks and such; this would have been devoid of sense. So he obviously cut his program into the stand-alone tool, and a MS Word add-in, which obviously DOES make a lot of sense, not only commercially, but also for the user, three quarter or more of prospects certainly preferring the MS Word add-in now.

From the stand-alone version, though, you can deduct that “major” (which means: paid) updates are quite frequent, so holding your version up-to-date could become quite onerous, and here it becomes evident that developer’s and users’ interests might not be entirely consistent in the long run.

Anyway, most/lots of SmartEdit’s functionality (both stand-alone and MS Word add-in) is about list making, which from a coding point of view is rather easy, but which for editing is not as wonderfully helpful as you might think beforehand; on the other hand, integrating more interactivity would mean lots more of coding effort, so…

From two very recent (Sept and Oct, 2014) discussions over at donationcodercom (just search their forum for “smartedit”) re SmartEdit I deduct that even now, some major = paid updates far from 1.0, the standalone version (and by extension, the MS Word spin-off) does not offer any functionality yet that will check repeated use of meaningful words (i.e. instead of using either a synonym now or rephrase the wording altogether) within a sentence or two or three consecutive sentences, i.e. some sort of “progressive vicinity search” for words (let alone radicals: such a function would signal, e.g. different forms of the same verb following each other, as presumably unwanted) from word 1 of a text down to the last word of its very last sentence, for each of them checking any other word / phrase in the current sentence and in the next two sentences for possible unwanted repetition - hint: the coding would not be too difficult here either, even though such functionality would indeed comprise some exception tables, and even individual ones for this subject range or that one (in medical texts, for example, it’s evident that the very same Latin expressions would be repeated over and over again, without such a tool harassing you with “Do you really want to use this term again here?” dialogs).

Also, it’s clear as day that such functionality would be so much more handy in an add-in tool like the one on offer here, than in a stand-alone tool, since only the add-in version (if correctly programmed that is) would enable this check functionality in real-time, i.e. within Word when you type the original text, and yes, mid-term, there should be some integrated learning functionality grasping and correctly processing your reactions to these “Do you really” pop-ups for further interaction with the user; for example, there could be an “intermediate” reaction from the program, which for some things would not pop the dialog up anymore, but would indicate some POSSIBLY unwanted repetition by highlighting the previous occurence of some word or phrase with some background color for some seconds or as long as your continuation of typing exceeds the sentence in curse.

Thus - but that’s just me - it seems to me that both/all versions of SmartEdit, of all possible, imaginable functions out there, lack the most important, needed and helpful one of them all, this word / phrase repetition check (the overall word frequency list (where the frequency of 1 is left out btw, or has it been integrated in the meanwhile?) very probably being a useless gimmick for most users (or then, tell me why we would need it in real life, except perhaps for 1 user in 100?).

As it stands, I thus do not see much interest for me in this program; for MS Word users, this would be quite different indeed once SmartEdit got the above-described, so much needed functionality. Btw, and as for text expanding, such functionality should be integrated not within an MS Word add-in, but within some dedicated tool, running in the background of ANY word processor and other text program (outliners, screenplay software, and so on), just as any text expanders does; technically, it would be quite easy for any such expansion / macro tool running in the background anyway to be spiced up this way: I just would need its own, dedicated memory for the very last 3 sentences, and it would do its repetition checked within that memorized text; accordingly, whenever you edit some other, non-current-writing text, the tool automatically would fetch the sentence you edit, plus two sentences before and after, and work on that text compound.

In a word, it’s clear as day that “run-time” text edit (i.e. in the curse of actual writing writing) is even more useful than after-the-spot editing (especially because the same corrections there are much less time-demanding than the same corrections here, afterwards, and also bec/of a learning tool not even presenting you repetitions and such anymore as possible candidates for correction once it will have identified them as presumably not unwanted), so the idea to integrate SmartEdit into MS Word is to be followed… but then, please offer more really helpful functionality! Wherever really needed functionality is left out, much less needed functionality becomes auxiliary and sometimes to the brink of irrelevant.