DevonThink 3 Evalutation

DevonThink supports reusability.

“We are drowning in information and starving for knowledge.” With this quote from John Naisbitt, Michael Malzahn begins his “DevonThink 3 Workbook.” The digital flood of data is crashing down on us from a variety of sources and inundating our end devices. We are faced with the seemingly insoluble task of making use of all these pieces of information.
With my Open Source Projekt “live-scripting”  I propagate the simultaneous creation of documentation while working on IT problems. But the benefit only comes when these solutions can be reused. The key is to quickly find similar solutions to a current problem.
The holy grail of searching across large document sets is full-text indexing, which is implemented in server applications like Solr or Elasticsearch, for example. On my Macbook, Spotlight search does a good job. Now I want to go a step further and have looked at the DevonThink application to do so. I summarize my experience in this article.

Figure 1: √úbersicht einer Suche in DevonThink.

Search in DevonThink is fast and powerful

DevonThink is essentially a program that makes large sets of files full-text indexed and searchable, in addition to providing robust synchronization and encryption. It is available for macOS and iOS.
The program is very complex and it took me almost a week to fathom the most important aspects.
DevonThink is a very powerful program. Its strengths lie in indexing and searching for files. This actually works at a speed I had never experienced before. The search expression can be very flexible, and the search space can be limited or extended individually. The results are highlighted and shown in a preview. This allows you to get a very quick overview of where the expression you are looking for appears.
Various inspectors, some of which use artificial intelligence, provide an even more detailed insight into the file being examined. DevonThink plays to its strengths with file formats such as PDF, or text-based files. These can be annotated or modified directly in the preview window. However, it is often better to start a specialized program, which is also possible directly from DevonThink. Unfortunately, IT specialists do not seem to be a target group for DevonThink. For example, no syntax highlighting is supported and special characters that frequently occur in programs are not valid search characters. It remains the grasp to find and egrep in the console still often necessary.

DevonThink contains a solid synchronization mechanism

Another important area of DevonThink is synchronization on the local network or to the cloud. In this case, the synchronization targets can be encrypted, which creates end-to-end encryption. In addition, DevonThink databases can be synchronized to multiple destinations in a very flexible way. These features make it possible to use cloud providers that you have limited trust in. Although I have not tested the synchronization myself, it is praised in many reviews and seems to be very robust.
Particularly exciting is the synchronization to iOS devices, but that doesn’t make much sense at the moment because the iOS version, DevonThink-To-Go 2.7 has only a limited feature set and is criticized in various forums and reports. An improved successor version has been announced, but unfortunately neither the features nor a release date are announced.

No one-size-fits-all program

Although the graphical interface of DevonThink has been modernized, it still reminds in parts very much of old Macintosh programs with various windows flying around freely. DevonThink is not a program from one source. In order to use the full range of functions, you have to have very deep knowledge, and you have to configure a lot and partly even program scripts. For example, email integration has been improved in version 3, but you still have to fiddle around with Apple scripts in the email program to get continuous updates. Stability is also not what you would want from a modern desktop program. It crashed or hung several times during my testing period.

Open formats secure the future of data

I really liked the openness of the program. All files are stored in databases that are directly accessible via the file system. DevonThink uses only open file formats. With the help of indexing, you can also search file trees in the file system without having to import the files.
There is an extensive manual that explains the most important aspects of the program. For the concrete configuration, however, this is often not sufficient, so that the well-maintained and moderated forum of DevonThink must be consulted more often. There is relatively little literature from other authors.

My private FAQ

In the following list I have summarized the questions that came up during the evaluation and had relevance for me.


Can I use upper and lower case in the search?
This doesn’t seem to work. It’s probably an optimization to not make the application’s memory footprint too large. However, within PDF documents you can select this.
Can I exclude groups from the search?
This is not possible via the advanced search dialog. But you can select the groups to be searched in the left sidebar.


Can entire file trees be imported?
Yes, this works very well with drag & drop. The files are copied into the database and can then be synchronized.
Can I edit files from the databases with external programs?
Yes, this works well, e.g. with Word files. But it is probably more problematic with programs that work on the file system, like Emacs or Git.


Can I index removable media and search in them?
Yes you can, for images and PDF files thumbnails are created. When the media is ejected the search still works. In the index the text is apparently present, but the texts cannot be displayed when the medium is ejected.
Are indexed files deleted from the file system?
Yes, this happens when they are placed in the trash in DevonThink and the trash is emptied. A selection dialog allows to delete only the objects inside the database.
How closely are indexed files connected to the file system?
Very closely, changes in the file are immediately shown in DT, even if they are open. They are immediately re-indexed. New files are immediately recognized and deleted files disappear from view.


How can I import Apple Notes?
This is easily done via the import dialog. All notes and attachments will be copied to DevonThink. The formatting will be broken in the process. Images and PDFs within notes are not displayed in DevonThink. However, all attachments are there as individual files.
Can I use the Apple Notes integration?
The integration is rather not usable. The notes break and even the search is better within Apple Notes.


How can I integrate emails?
There are three ways to integrate emails. The easiest is via the sidebar and the import setting (icon). All accounts in Apple Mail are then displayed. You can select individual folders, or the entire account. Unfortunately, I could not test the import finally, because the evaluation license did not allow it.
Can I reference emails from external programs?
This is possible with “Copy reference” from the context menu. The references are clickable and can be pasted into RTF documents and notes. They can also be pasted into Orgmode files. But for this I needed an adjustment in the .spacemacs file. The export of these links to HTML did not work. But it works in principle. I would have to find or write a module for this.
Are the E-Mail archives updated?
This is not done automatically. You have to update per mail account in the sidebar->import view first and then archive. I guess you can automate this with scripts. In my tests the process does not work reliably. I sent mail between two accounts. I had to refresh several times. DevonThink then mixed up the “Sent” and “Received” mailboxes.
How long does it take to import the mails daily?
If the Apple mail script is used it can take a long time. With the DevonThink Mail plug-in it is much faster.
How do I know that the mail plug-in is installed?
In menu of Apple-Mails exists the entry Menu->E-Mail->”Add to DevonThink 3″. To do this, it must first be installed via DevonThink->”Install additional modules” and then activated in Mail (Mail->Preferences->General->Manage plug-ins).
How to automate email archiving?
Unfortunately, this is complicated. First, all emails are archived via Sidebar->Import. Then a rule is created in Mail. (Settings->Rules). As condition you choose e.g. Received or Sent less than 7 days ago. As action you choose “Run Apple Script”. Then copy the script “Mail Rule – File messages & attachments hierarchically” into a new script and edit the target database from DT3. Afterwards all new emails are archived automatically. Unfortunately the script does not process sent mails.


Can whole web pages be loaded (crawler)?
Yes, but the tool does not necessarily provide satisfactory results. Especially with dynamic web pages. I import my local static website (orgweb) better directly from the webroot via drag and drop over the file system. The result is very good. Navigation and search within the integrated browser works very well.


Can contacts be imported?
Yes, it works quite well. E.g. as a vcard. Unfortunately the card does not contain pictures and the notes section of a contact is not transferred.
Should I import contacts?
Maybe to find people quickly. But it does not replace the contacts app.
Are the links in the vcards active?
Yes, you can click to start an email, send a message or start a Facetime call.


Is it possible to create an item link to a page in Word?
This is supposed to work, but it did not work in a quick test. Although the item link contains “line=200” as parameter, the document is opened at the beginning. In addition, only a narrow section of the document is shown, even if you zoom in, just like with the Apple preview. Word is not a first class citizen in DevonThink.
Is it possible to create an item link to a PDF page?
Yes, this works very well. A link is created from the context menu with “Link to page”. The document will then be opened at this page.
Is it possible to create an item link to an RTF paragraph?
Yes, this works very well. The document is opened at the appropriate place and the paragraph is marked.
How stable is DevonThink?
Not super stable. It crashed when annotating a PDF. The left sidebar has problems updating the view, for example when deleting or creating groups. When viewing web archives it has hung several times.
Is it possible to enlarge the font?
Yes, there are “Text sizes for display” and “Text size for sidebar” in the settings. Unfortunately this is not consistent. For example, menus in the advanced search dialog are not scaled and are barely readable on a high resolution screen.
Is it possible to use tabs?
This is only possible for documents. With Data->Open in Tabs + ALT key (Modifiyer Key) selected documents are opened in multiple tabs. But you can also add tabs, the selected document will be opened in the active tab. You can launch multiple DevonThink main windows, but they cannot be arranged in tabs.


DevonThink is a powerful program with rough edges that makes researching in a large search space very easy, but it can also take a lot of time to configure, design your own workflow and solve individual problems.



Using Live-Scripting to set up Labs on AWS

My Emacs based live-scripting approach can be used to explore AWS technology. I have demonstrated this with an example of a basic EC2 lab.
In this lab, I set up a simple EC2 instance with the command line interface. I use the EC2 “user data” to set up an Apache Web server, and I initialize the index page with meta-data of this instance.
The following diagram gives an overview of this lab.

The documentation and a code are stored in my project aws-labs on GitHub. If you have Emacs configured for live-scripting, this lab can be setup in less then five minutes, which is impressive. You can learn more about the live-scripting approach on my project live-scripting on GitHub.