Tinybase

Mike_Sanders

@ctietze said:
The bolt example reminds me of a sociological study that stuck with me: "Crime and Punishment in the Factory: The Function of Deviancy in Maintaining the Social System"

https://onwork.edu.au/bibitem/1963-Bensman,Joseph-Gerver,Israel-Crime+and+Punishment+in+the+Factory+The+Function+of+Deviancy+in+Maintaining+the+Social+System/

I bet you find copies of that to read. The crux is that even though drilling new threading into bolt holes is officially forbidden and punished, there's a rather complex informal agreement on when and how to re-thread holes, if needed.

Hey-hey Christian

That article is fascinating!

Mike_Sanders

Okay, quick update: I'm about to enter the last leg of the jouney with Tinybase. Here's where I'm at...

Builds for Windows, Linux, OpenBSD, NetBSD, FreeBSD, Minix & a preliminary build for Mac (more testing needs to be done there).

Additionally, Tinybase in both REPL & scripting modes now runs over a networked connection using any of SSH/Telnet/RSH & of course on a USB stick or plain old computer.

All configuration is now outside of userspace & contained within an external file parsed at runtime as shown next:

# tinybase color table

# default 0   green  3   magenta 6
# black   1   yellow 4   cyan    7
# red     2   blue   5   white   8

# tinybase profile settings

BLOCK-COLOR    7  #  0 to 8 see color table
HEADER-COLOR   5  #  0 to 8 see color table
NUMBERS-COLOR  2  #  0 to 8 see color table
PRECISION      2  #  0 to 8 decimal places
CURSOR         2  #  0 default, 1 underline, 2 bar, 3 block
ENCODING       0  #  0 none, 1 xor-byte, 2 caesar's revenge, 3 rot13
SCAN-ENGINE    0  #  0 literal, 1 phonetic, 2 regex
INDEX          2  #  0 unsorted, 1 sorted A-Z, 2 sorted by frequency
LINE-NUMBERS   1  #  0 off, 1 on
SHELL-ACCESS   1  #  0 off, 1 on

# eof

Conversely, if Tinybase needs to be ran in a more constrained or embedded environment, or even if you don't want a config file, one can instead simply edit the last 11 bytes of the Tinybase binary itself as shown below:

a leading dash (-) means 'dont' read the following digits: -7500000011

a leading plus (+) means 'do' read the following digits:   +7500000011

Manual explains all the details. Also 3 encoding schemes using the following streaming ciphers:

xor()

caesars-revenge() (footnote below)

rot13()

Ever onwards!

https://forum.zettelkasten.de/discussion/2375/caesars-revenge

Mike_Sanders

Re: I bet you find copies of that to read...

Piecing it together bit by bit.

Mike_Sanders

Hooray! Macros done.

Given a tagged block with this content:

Tinybase expands defined macros 'in place' ([M8] & [M9] are not defined & thus rendered literally) within the affected block...

The macros are defined in the user's config file & parsed when Tinybase initializes...

A good day

Mike_Sanders

Okay, a final tweak to the metrics.

Decided to calculate a file's Flesch Score... This should be considered subjective & perhaps even archaic (see footnote), nevertheless its an interesting metric.

/*

readability...

90-100  very easy
80-90   easy
70-80   fairly easy
60-70   plain english
50-60   fairly difficult
30-50   difficult
 < 30   very difficult

*/

Enough coding - I need to get back to work on the manual (does it ever end?)

• In 1943, Rudolf Flesch published his PhD dissertation, Marks of a Readable Style, which included a readability formula to predict the difficulty of adult reading material.

ctietze

If you use this score now, I'd also be interested in how you perceive its effects on your writing. Do you begin to edit notes later? Do you ignore it? Etc.

Mike_Sanders

Hi Christian. I'm not sure just yet. I'm still studying/learning/thinking. Some notes to share...

Mike_Sanders

@ctietze said: If you use this score now, I'd also be interested in how you perceive its effects on your writing. Do you begin to edit notes later? Do you ignore it? Etc.

You know, yes, I think I will use these Christian. At least tentatively. Very interesting metrics on the whole. Besides Flesch, I've also added: ARI, Lix, & Coleman-Liau.

Here's the metrics output from the puzzle data you've seen before upthread, as well as another data file that simply contains some quick notes. The results are striking imo...

Just whistle if you want to peek at the formulas.

metrics...

file      puzzles.txt
size      24.9KB
modified  2022.12.06-15:04:00
crc8      0xb8
lines     857
blocks    50 [ignored: 0]
tags      151 [per block avg: 3.02]

ari score...

characters   14,671
words        9,926
sentences    5
readability  professor [ages 24+]

lix score...

words        9,926
long words   342
sentences    5
readability  99.50 [very difficult]

flesch-kincaid score...

syllables    4,544
words        9,926
sentences    5
readability  30.00 [very difficult]

coleman-liau score...

letters      14,514
words        9,926
sentences    5
readability  -7.00 [unreadable...]

-- end output ----------------------------------------------------------------

metrics...

file      papers.txt
size      140.3KB
modified  2022.12.08-20:10:31
crc8      0xcf
lines     4,517
blocks    38 [ignored: 0]
tags      92 [per block avg: 2.42]

ari score...

characters   83,691
words        20,196
sentences    1,752
readability  3rd grade [ages 7-9]

lix score...

words        20,196
long words   3,795
sentences    1,752
readability  30.32 [easy]

flesch-kincaid score...

syllables    26,207
words        20,196
sentences    1,752
readability  85.00 [easy]

coleman-liau score...

letters      77,240
words        20,196
sentences    1,752
readability  4.00 [5th grade & below]

Mike_Sanders

For a number of years, I've been 'plotting' my notes with a graphics app & thought after adding to this thread: Why not build my own charts in plain text?

Had to right justify the index of tag names on the left, & yet, left justify the bargraph on the right (that's a mouthful) so the rows below align with the rows above.

Still, works pretty well really...

Mike_Sanders

Okay, have proper justification now...

Mike_Sanders

Ok, crude tree in place. I can map parent to child relationships & sibling to sibling relationships, a type of node vector, witness...

But more distant affinities (sorry I'm grappling with the terminology as I'm self-taught) such as a relation/correlation between say, block 1 tag 'code' & block 82 tag 'code', I'm still brooding over. Hope I can nail this. Must continue to mull this over more.

Never say die...

ctietze

What, no Unicode box drawing characters?!?!
https://en.wikipedia.org/wiki/Box-drawing_character
https://unicode-table.com/en/blocks/box-drawing/

The tree command does this really nicely: http://mama.indstate.edu/users/ice/00Tree.html
ASSCII fallback: http://mama.indstate.edu/users/ice/tree/

Mike_Sanders

Chuckle I knew you were going to say that! And yes, the 7bit tree is butt-ugly. Thanks or the links Christian, I very much appreciate it, confirms I'm on the right track.

Wound up using ANSI/Hex encoding. Snippet below for those interested in these things.

/*
tags: code, box, ansi, c, sh

hex encoded ansi box characters for c...

  usage: printf("%s\n", VL);

  #define RC "\e(0\x75\e(B" // 185 right cross
  #define VL "\e(0\x78\e(B" // 186 vertical line
  #define RB "\e(0\x6a\e(B" // 188 right bottom corner
  #define RT "\e(0\x6b\e(B" // 187 right top corner
  #define LB "\e(0\x6d\e(B" // 200 left bottom corner
  #define LT "\e(0\x6c\e(B" // 201 left top cornet
  #define BC "\e(0\x76\e(B" // 202 bottom cross
  #define TC "\e(0\x77\e(B" // 203 top cross
  #define LC "\e(0\x74\e(B" // 204 left cross
  #define HL "\e(0\x71\e(B" // 205 horizontal line
  #define MC "\e(0\x6e\e(B" // 206 middle cross

hex encoded ansi box characters for shell scripts...

  usage: echo -e $VL

  RC="\e(0\x75\e(B" # 185 right cross
  VL="\e(0\x78\e(B" # 186 vertical line
  RB="\e(0\x6a\e(B" # 188 right bottom corner
  RT="\e(0\x6b\e(B" # 187 right top corner
  LB="\e(0\x6d\e(B" # 200 left bottom corner
  LT="\e(0\x6c\e(B" # 201 left top cornet
  BC="\e(0\x76\e(B" # 202 bottom cross
  TC="\e(0\x77\e(B" # 203 top cross
  LC="\e(0\x74\e(B" # 204 left cross
  HL="\e(0\x71\e(B" # 205 horizontal line
  MC="\e(0\x6e\e(B" # 206 middle cross
*/

Mike_Sanders

Hyperlink support ready (if your terminal supports it)...

Mike_Sanders

I'm left gob-smacked looking at some of the metrics tinybase is producing. By percentage I've always assumed the largest amount of tags in any of my data files were labeled 'code' & while that is certainly true, it doesn't tell the whole story. Now with the 'node' (tree) view, new patterns are surfacing I was not previously cognizant of, chiefly orphaned tags (not sure, but I believe Will coined that term). And 'work-horse' nodes (tagged blocks with a multiplicity of aliases)...

Really amazing to be able to quantify one's data in differing contexts. Almost like having fresh eyes. Well, back to work, defining the grammar I'm using in the app's manual. Exciting times for me

ctietze

Happy to see the way more sexy characters being used

I'm left gob-smacked looking at some of the metrics tinybase is producing. By percentage I've always assumed the largest amount of tags in any of my data files were labeled 'code' & while that is certainly true, it doesn't tell the whole story. Now with the 'node' (tree) view, new patterns are surfacing I was not previously cognizant of, chiefly orphaned tags (not sure, but I believe Will coined that term). And 'work-horse' nodes (tagged blocks with a multiplicity of aliases)...

Can you explain what we're looking at in the picture some more? E.g. I've scanned the screenshot for 'code', but didn't find it anywhere, so I'm not sure how "the largest amount of tags [...] were labeled 'code'" and "new patterns are surfacing"

Mike_Sanders

Yes, I'll try

You are correct, there is no code tag (which surprised me because so many of my zettel are about code). This is because I've created a new model Christian, a new map, a new way to view my notes. Its hard to explain, but here it is nevertheless:

tag is to zettel as node is to kasten...

I know hard to understand...

Mike_Sanders

Wordwrap complete...