Please help me move to BibDesk

amahabal

Hello!

I am a Bookends user, and I have a nice collection of PDFs associated with references. I was able to dump the references into a *.bibtex file and import it into bibdesk, but I don't know how I can attach those PDFs to references in BibDesk. Does someone here know?

Also, bookends supports what they call, I think, "floating citations", where pressing alt-alt in The Archive brings up a small window where I can search a reference and it pastes, for example, [@Smith1976] into the Archive. This too I'd love to replicate, but if not available, I can script it easily enough using an unholy mix of Python and Keyboard Maestro and Alfred.

Why I want to switch: Bookends isn't very git-friendly; they produce a binary file containing the references instead of a text file like bibtex.

Many thanks!

Andy

Here's a tip that may help (I haven't tested this but I'm a long-time BibDesk user so I know how it should work):

1. Make sure each of the entries in your BibTeX file has a local-url field with the path to the PDF for that entry.
2. In the Default Field Preference Pane in BibDesk's Preferences/Settings, you could select (check) all of the following checkbox settings:

• Use linked files and URLs
• Automatically convert File and URL fields
• Remove converted Local File fields

3. In the same Preference Pane, look at the "Preferred local file format" menu, and select the format that corresponds to how your local-url field is formatted.
4. When these settings are selected, after you quit BibDesk and reopen your BibTeX file, BibDesk will convert a local URL in the local-url field into a special Base64-encoded string in a Bdsk-File-1 field, which is how BibDesk stores its links to local files.

If you don't have, or can't easily create, local-url fields, then the manual way to attach files to BibDesk entries is to drag the file from, for example, a Finder window onto the publication in the main table of publications in BibDesk and let go when the publication is selected, or drag the file into the "Drop Files Here" area in the publication editor window.

Will

@amahabal, I'm not familiar with Bookends. I started with BibDesk but switched to Zotero a couple of years ago. Many current and former members of this forum recommended Zotero. I had difficulty getting some aspects of my "unholy mix of Python and Keyboard Maestro" to work with BibDesk, so I switched. I can't remember if it was syncing with multiple Macs, some other interoperability, or the scripting.

This will create a Frankenstein version of "floating citations."
I have a group of Keyboard Maestro macros that take a .bib file and read it, producing a type-ahead search that places either an in-line citation or a full citation in the MLA format, which my instructors require. Fiddley to set up, but it has worked great since I got the bugs worked out. It might work with Bibdesk because the insert citations only require a .bib file which BibDesk can produce.

I didn't realize what a monster Igor and I had created until I went to attach the files to this message.

To export a .bib file whenever changes are made, Zotero requires the installation of the Better BibTex extension, which comes with other essential features.

I use a Python script to read the .bib file and extract the pieces needed for citations.It's found here.

Here are three Keyboard Maestro macros.
Build listOfBibEntries
Add Full Citation
Add Citation

What a mess! But it works. If you use any of this, you'll have to change the triggers, the file locations, and the format of the reference full citation if you want something else besides what I've configured.

Andy

I prefer BibDesk to Zotero, but I actually run both of them at the same time because there is a great Zotero add-on called Zot2Bib that allows scraping references from a web browser (with the Zotero Connector add-on) to BibDesk via Zotero. (This doesn't duplicate items in both apps; it just funnels data through Zotero to BibDesk, and Zotero's library remains empty.) BibDesk's internal search groups are great too, and most of the books that I import come through library catalog search groups in BibDesk, but most non-book publications I scrape via the Zotero button in my web browser. So don't abandon BibDesk for Zotero if all you want is Zotero's web scraping, since you can have that and BibDesk with Zot2Bib. BibDesk does web scraping in its internal web groups, but I've never used it because I want all my web history to be in my primary web browser, so I use Zot2Bib.

amahabal

Thanks, @Andy and @Will, for your generous help!

I have followed your suggestions by adding a local-URL field to entries (using the Python module BibTeX parser, guessing the filename using the bookends format (that is, author_year, author1_&_author2_year or author1_et_al_year, as the case may be) and attaching the file if it was present.

I assume that the Bdsk-file-1 is not helpful to put into git (it won't be the same on a different machine?!?), so removing that.

I will do the floating citations (following suggestions from @Will) when I am not 90% asleep...

Andy

@amahabal said:

I assume that the Bdsk-file-1 is not helpful to put into git (it won't be the same on a different machine?!?), so removing that.

If you don't want the Bdsk-file-1 field, you shouldn't select "Automatically convert File and URL fields" and "Remove converted Local File fields" as I suggested above. Leave those options unchecked. You probably already figured that out. Perhaps "Use linked files and URLs" needs to be unchecked too; I don't know. That's beyond my experience.

I don't put my main BibTeX file under version control; mine is so big that it would be a waste of disk space and waste of effort to try to track changes. For those reasons, I suggest not putting your main BibTeX file in git unless you have some other good reason for doing so that outweighs the disadvantages.

Andy

@Andy said:

I don't put my main BibTeX file under version control; mine is so big that it would be a waste of disk space and waste of effort to try to track changes. For those reasons, I suggest not putting your main BibTeX file in git unless you have some other good reason for doing so that outweighs the disadvantages.

I'm not a git expert, but I can give a clear example of what I mean when I say that putting my main BibTeX file under version control with git would be a waste of disk space: Philosophy professor Brad Weslake keeps his BibDesk database in this public git repository. I cloned the repository to my hard drive to take a look at it, and I discovered that was a big mistake: although his BibTeX file was only 17MB, the whole git repository occupied 337MB on my hard drive. My main BibTeX file is larger than Weslake's, so if I had put it in git years ago and committed changes as often as he did, the resulting git repository would occupy a huge amount of disk space—and unnecessarily for my purposes, since I don't need more information about changes to the file than what BibDesk's date-created and date-modified fields for each entry already provide.

amahabal

@Andy said:
@Andy said:

I don't put my main BibTeX file under version control; mine is so big that it would be a waste of disk space and waste of effort to try to track changes. For those reasons, I suggest not putting your main BibTeX file in git unless you have some other good reason for doing so that outweighs the disadvantages.

I'm not a git expert, but I can give a clear example of what I mean when I say that putting my main BibTeX file under version control with git would be a waste of disk space: Philosophy professor Brad Weslake keeps his BibDesk database in this public git repository. I cloned the repository to my hard drive to take a look at it, and I discovered that was a big mistake: although his BibTeX file was only 17MB, the whole git repository occupied 337MB on my hard drive.

I am 99% sure that they are doing something wrong and that something wrong is the pesky Bdsk-file business. Here is an example from their change, which is too big for GitHub to show me: https://github.com/etc/philosophy-bibliography/commit/cc53d8c9f49eef179807b84c692217dfb8864ad5 As you can see this has 7640 additions and 7513 deletions. I suspect that they added maybe 15 citations in this change. I hypothesize that two computers are involved here. The previous change was on computer 1, the second on computer 2, with entirely different Bdsk links. So those 15 additions seemed to git like most of the file changing rather than one chunk. They have made 500 commits, and many commits are enormous, requiring much space. With just additions, the deltas should look more like this (although even here, the delta is bigger than it needs to be!) https://github.com/etc/philosophy-bibliography/commit/dfe72cc1374659a314f07a8c6e42303e988f0206

My main BibTeX file is larger than Westlake's,

Just out of curiosity, how many entries do you have?

Best,
Abhijit

Andy

@amahabal said:

Just out of curiosity, how many entries do you have?

A lot. Let's just say, I can attest that BibDesk works fine with tens of thousands of entries. A long time ago, as my main BibTeX file grew, I adopted two BibTeX files in daily use: the main file, and an inbox file where new entries stay until I have time (usually every month or so, unless I need the entries to be in the main file sooner) to review a batch of entries to correct common data quality problems using a long list of smart groups created to catch particular problems. Then I use a text editor to copy and paste the new entries into the main file. I had to use two files because I found that having a long list of smart groups in a BibDesk file with many entries (say, over ten thousand? I can't remember the quantity exactly) doesn't work well because the smart groups take too long to update, processing so many entries. This won't be a problem as long as your BibTeX file is fairly small.

Andy

The problem with smart groups that I mentioned in the last comment doesn't happen with static groups and field groups. Long lists of static groups and field groups work perfectly in a large BibDesk file because they are not computationally intensive.