Blog

Using Live-Scripting to install Oracle Service Bus

Since a couple of years, I am already using the live-scripting approach in customer projects. I received a lot of positive feedback, especially for the well-structured and precise documentation in HTML format, that this technique produces. I have been using it in various situations, and it proved to be useful especially for the installation and configuration of Oracle products like Fusion Middleware, Oracle Service Bus and Forms and Reports.
In my new GitHub project, I introduce the installation of a simple OSB architecture using the live-scripting approach.

Installation Process of Oracle Service Bus using Live-Scripting

Oracle Service Bus (OSB) is widely utilized, boasting a substantial installation base. Although new projects are currently rare, its complex environment prevails. Expertise is often sought after for version migrations and the integration of new solutions. In the context of Oracle Service Bus (OSB) installation, live-scripting is a powerful approach that combines the execution of command-line operations with detailed documentation to create a reliable and well-documented installation process. Live-scripting benefits OSB installation in the following ways:

Reproducibility: Live-scripting ensures that the OSB installation process is reproducible, meaning that anyone following the documented steps can achieve the same results. This is particularly important in enterprise environments, where consistency and reliability are paramount. Even if the installation needs to be repeated months or years later, the documented process guarantees the same outcome.

Documentation: The process of live-scripting creates precise documentation that captures each step involved in the process. This documentation goes beyond simple command lists; it includes explanations, diagrams, attachments and solutions to potential issues that may arise. This detailed documentation is invaluable for troubleshooting, auditing, and knowledge transfer within an organization.

Quality Assurance: Live-scripting serves as a form of quality assurance. The fact that the documented process has been tested and verified during writing, ensures that the documented solution is reliably working and the chances of errors and misconfigurations is minimal.

Knowledge Sharing: The live-scripting approach makes knowledge sharing much more accessible. Team members can easily follow the step-by-step instructions, regardless of their level of expertise. Even if they don’t want to use Emacs, the solution can be reproduced using a copy-and-paste approach. It accelerates the training process for new team members and can help disseminate best practices across the organization.

Flexibility and adaptability: Live-scripting isn’t limited to a one-size-fits-all approach. It can be adapted and extended as needed. Users can modify the documented process to accommodate unique requirements or make improvements over time without sacrificing the stability and reliability of the installation.

Efficiency: Live-scripting streamlines the installation process by providing a clear and efficient path from start to finish. It minimizes the time and effort required to set up OSB, making it more accessible for both experienced and novice users.

In contrast to a fully automated setup, which is often used in large OSB installations, encompassing hundreds of domains and environments, the approach proposed by live-scripting does not require writing, test and maintain automation code. Live-scripting is best used in situations where only a limited number of environments are required. It ranges between a classical manual approach on the one hand and a fully automated solution on the other. In summary, live-scripting is a powerful methodology in the context of command-line-centered work, especially during OSB installation. It ensures a reproducible, well-documented, and reliable installation process, which is crucial in complex enterprise environments. This approach not only guarantees quality and consistency but also promotes knowledge sharing, adaptability, and efficiency in managing installations.

If I could raise your interest for this subject, please take a look at my GitHub Project live-scripting_osb, which demonstrates the advantages of the live-scripting approach in the context of an OSB installation. If you would like to use the same approach in your own project, you might also be interested in my commercial offerings for OSB projects.

 

DevonThink 3 Evalutation

DevonThink supports reusability.

“We are drowning in information and starving for knowledge.” With this quote from John Naisbitt, Michael Malzahn begins his “DevonThink 3 Workbook.” The digital flood of data is crashing down on us from a variety of sources and inundating our end devices. We are faced with the seemingly insoluble task of making use of all these pieces of information.
With my Open Source Projekt “live-scripting”  I propagate the simultaneous creation of documentation while working on IT problems. But the benefit only comes when these solutions can be reused. The key is to quickly find similar solutions to a current problem.
The holy grail of searching across large document sets is full-text indexing, which is implemented in server applications like Solr or Elasticsearch, for example. On my Macbook, Spotlight search does a good job. Now I want to go a step further and have looked at the DevonThink application to do so. I summarize my experience in this article.

Figure 1: Übersicht einer Suche in DevonThink.

Search in DevonThink is fast and powerful

DevonThink is essentially a program that makes large sets of files full-text indexed and searchable, in addition to providing robust synchronization and encryption. It is available for macOS and iOS.
The program is very complex and it took me almost a week to fathom the most important aspects.
DevonThink is a very powerful program. Its strengths lie in indexing and searching for files. This actually works at a speed I had never experienced before. The search expression can be very flexible, and the search space can be limited or extended individually. The results are highlighted and shown in a preview. This allows you to get a very quick overview of where the expression you are looking for appears.
Various inspectors, some of which use artificial intelligence, provide an even more detailed insight into the file being examined. DevonThink plays to its strengths with file formats such as PDF, or text-based files. These can be annotated or modified directly in the preview window. However, it is often better to start a specialized program, which is also possible directly from DevonThink. Unfortunately, IT specialists do not seem to be a target group for DevonThink. For example, no syntax highlighting is supported and special characters that frequently occur in programs are not valid search characters. It remains the grasp to find and egrep in the console still often necessary.

DevonThink contains a solid synchronization mechanism

Another important area of DevonThink is synchronization on the local network or to the cloud. In this case, the synchronization targets can be encrypted, which creates end-to-end encryption. In addition, DevonThink databases can be synchronized to multiple destinations in a very flexible way. These features make it possible to use cloud providers that you have limited trust in. Although I have not tested the synchronization myself, it is praised in many reviews and seems to be very robust.
Particularly exciting is the synchronization to iOS devices, but that doesn’t make much sense at the moment because the iOS version, DevonThink-To-Go 2.7 has only a limited feature set and is criticized in various forums and reports. An improved successor version has been announced, but unfortunately neither the features nor a release date are announced.

No one-size-fits-all program

Although the graphical interface of DevonThink has been modernized, it still reminds in parts very much of old Macintosh programs with various windows flying around freely. DevonThink is not a program from one source. In order to use the full range of functions, you have to have very deep knowledge, and you have to configure a lot and partly even program scripts. For example, email integration has been improved in version 3, but you still have to fiddle around with Apple scripts in the email program to get continuous updates. Stability is also not what you would want from a modern desktop program. It crashed or hung several times during my testing period.

Open formats secure the future of data

I really liked the openness of the program. All files are stored in databases that are directly accessible via the file system. DevonThink uses only open file formats. With the help of indexing, you can also search file trees in the file system without having to import the files.
There is an extensive manual that explains the most important aspects of the program. For the concrete configuration, however, this is often not sufficient, so that the well-maintained and moderated forum of DevonThink must be consulted more often. There is relatively little literature from other authors.

My private FAQ

In the following list I have summarized the questions that came up during the evaluation and had relevance for me.

SEARCH

Can I use upper and lower case in the search?
This doesn’t seem to work. It’s probably an optimization to not make the application’s memory footprint too large. However, within PDF documents you can select this.
Can I exclude groups from the search?
This is not possible via the advanced search dialog. But you can select the groups to be searched in the left sidebar.

IMPORT

Can entire file trees be imported?
Yes, this works very well with drag & drop. The files are copied into the database and can then be synchronized.
Can I edit files from the databases with external programs?
Yes, this works well, e.g. with Word files. But it is probably more problematic with programs that work on the file system, like Emacs or Git.

INDEXING

Can I index removable media and search in them?
Yes you can, for images and PDF files thumbnails are created. When the media is ejected the search still works. In the index the text is apparently present, but the texts cannot be displayed when the medium is ejected.
Are indexed files deleted from the file system?
Yes, this happens when they are placed in the trash in DevonThink and the trash is emptied. A selection dialog allows to delete only the objects inside the database.
How closely are indexed files connected to the file system?
Very closely, changes in the file are immediately shown in DT, even if they are open. They are immediately re-indexed. New files are immediately recognized and deleted files disappear from view.

APPLE NOTES

How can I import Apple Notes?
This is easily done via the import dialog. All notes and attachments will be copied to DevonThink. The formatting will be broken in the process. Images and PDFs within notes are not displayed in DevonThink. However, all attachments are there as individual files.
Can I use the Apple Notes integration?
The integration is rather not usable. The notes break and even the search is better within Apple Notes.

EMAIL

How can I integrate emails?
There are three ways to integrate emails. The easiest is via the sidebar and the import setting (icon). All accounts in Apple Mail are then displayed. You can select individual folders, or the entire account. Unfortunately, I could not test the import finally, because the evaluation license did not allow it.
Can I reference emails from external programs?
This is possible with “Copy reference” from the context menu. The references are clickable and can be pasted into RTF documents and notes. They can also be pasted into Orgmode files. But for this I needed an adjustment in the .spacemacs file. The export of these links to HTML did not work. But it works in principle. I would have to find or write a module for this.
Are the E-Mail archives updated?
This is not done automatically. You have to update per mail account in the sidebar->import view first and then archive. I guess you can automate this with scripts. In my tests the process does not work reliably. I sent mail between two accounts. I had to refresh several times. DevonThink then mixed up the “Sent” and “Received” mailboxes.
How long does it take to import the mails daily?
If the Apple mail script is used it can take a long time. With the DevonThink Mail plug-in it is much faster.
How do I know that the mail plug-in is installed?
In menu of Apple-Mails exists the entry Menu->E-Mail->”Add to DevonThink 3″. To do this, it must first be installed via DevonThink->”Install additional modules” and then activated in Mail (Mail->Preferences->General->Manage plug-ins).
How to automate email archiving?
Unfortunately, this is complicated. First, all emails are archived via Sidebar->Import. Then a rule is created in Mail. (Settings->Rules). As condition you choose e.g. Received or Sent less than 7 days ago. As action you choose “Run Apple Script”. Then copy the script “Mail Rule – File messages & attachments hierarchically” into a new script and edit the target database from DT3. Afterwards all new emails are archived automatically. Unfortunately the script does not process sent mails.

LOADING WEB PAGES

Can whole web pages be loaded (crawler)?
Yes, but the tool does not necessarily provide satisfactory results. Especially with dynamic web pages. I import my local static website (orgweb) better directly from the webroot via drag and drop over the file system. The result is very good. Navigation and search within the integrated browser works very well.

CONTACTS

Can contacts be imported?
Yes, it works quite well. E.g. as a vcard. Unfortunately the card does not contain pictures and the notes section of a contact is not transferred.
Should I import contacts?
Maybe to find people quickly. But it does not replace the contacts app.
Are the links in the vcards active?
Yes, you can click to start an email, send a message or start a Facetime call.

GENERAL

Is it possible to create an item link to a page in Word?
This is supposed to work, but it did not work in a quick test. Although the item link contains “line=200” as parameter, the document is opened at the beginning. In addition, only a narrow section of the document is shown, even if you zoom in, just like with the Apple preview. Word is not a first class citizen in DevonThink.
Is it possible to create an item link to a PDF page?
Yes, this works very well. A link is created from the context menu with “Link to page”. The document will then be opened at this page.
Is it possible to create an item link to an RTF paragraph?
Yes, this works very well. The document is opened at the appropriate place and the paragraph is marked.
How stable is DevonThink?
Not super stable. It crashed when annotating a PDF. The left sidebar has problems updating the view, for example when deleting or creating groups. When viewing web archives it has hung several times.
Is it possible to enlarge the font?
Yes, there are “Text sizes for display” and “Text size for sidebar” in the settings. Unfortunately this is not consistent. For example, menus in the advanced search dialog are not scaled and are barely readable on a high resolution screen.
Is it possible to use tabs?
This is only possible for documents. With Data->Open in Tabs + ALT key (Modifiyer Key) selected documents are opened in multiple tabs. But you can also add tabs, the selected document will be opened in the active tab. You can launch multiple DevonThink main windows, but they cannot be arranged in tabs.

Conclusion

DevonThink is a powerful program with rough edges that makes researching in a large search space very easy, but it can also take a lot of time to configure, design your own workflow and solve individual problems.

 

 

Using Live-Scripting to set up Labs on AWS

My Emacs based live-scripting approach can be used to explore AWS technology. I have demonstrated this with an example of a basic EC2 lab.
In this lab, I set up a simple EC2 instance with the command line interface. I use the EC2 “user data” to set up an Apache Web server, and I initialize the index page with meta-data of this instance.
The following diagram gives an overview of this lab.

The documentation and a code are stored in my project aws-labs on GitHub. If you have Emacs configured for live-scripting, this lab can be setup in less then five minutes, which is impressive. You can learn more about the live-scripting approach on my project live-scripting on GitHub.

Introduction Video for Live-Scripting

Live-Scripting is a documentation centric working style for IT professionals that promotes reuse and sharing of proven solutions. In my previous video, I already demonstrated how to work on the command line and document it at the same time. In the video “Live-Scripting Introduction” I take a broader perspective and address the problem of the growing amount of knowledge that needs to be understood, digested, documented and shared. Companies often struggle with sharing of knowledge and solutions because documentation is a time-consuming and often unpleasant task. On the other hand, even experienced engineers repeat analysis of the similar problems because they have simply forgotten the details of  the work that they have done weeks or month ago. Fighting against the forgetting curve is a prevalent challenge.

The Live-Scripting approach aims at making the process of creating, sharing and reusing documentation smooth and effective.  

 

Video Clip on basic Live-Scripting

In my new short video, I introduce the basic idea behind my live-scripting open source project which I host on GitHub. Live-Scripting is an approach that combines command line centric work, documentation, and sharing. This video clip demonstrates how smooth the process of command line work and documentation can be. Using the example of retrieving information from remote machines and creating a report, this clip introduces how easy this technique is and gives an idea, how powerful it can be. Emac users can use these basic techniques directly with only little configuration. The necessary steps are described on the project -website in the chapter “Basic Live-Scripting”. This is even possible if Emacs is used in a terminal without GUI.

Emacs Spreadsheets Demo

In Emacs Org mode, you can create spreadsheets and also make calculations. The Spreadsheet tool is of course not as powerful as Excel, but it is enough for quick calculations. For example, you can use it to calculate the cost of a trip. In my Youtube channel, I published a short video demonstrating this feature. It compares two scenarios of a business trip regarding their costs.

Web Single Sign-On with SAML 2.0

While SAML is already widely used in the industry, the configuration within WebLogic
Server is complex and in most companies not part of the regular routine. We want to take a look at a simple SAML example that was published in an article by VikrantSawant in 2007. This former example demonstrates a Web
SSO scenario using SAML 1.1 in WebLogic Server 9.2.
We want to upgrade this example, using SAML 2.0 in WebLogic Server 12.1.3.
This is a tutorial in which we will walk through all the necessary steps to set up and run the SAML 2.0 example. This includes the installation and configuration of WebLogic server, creation of two WebLogic server domains, installation of the test applications and configuration of the identity provider and service provider domains. To provide a comprehensive overview, the separate tutorial steps are summarized in mind map diagrams. The tutorial comprises a Service Provider initiated flow and an Identity Provider initiated flow, which both will be demonstrated during the testing steps. As an addition, the tutorial demonstrates the usage of the WebLogic feature “virtual user”.

The tutorial was developed and tested on a Windows 7 machine. A zip package containing all necessary files is provided at the tutorial website. This also includes a text file with a set of windows commands to help to set up the domains and user configurations. We expect the tutorial to run also on Linux or any other platform supported by WebLogic server, although this was not tested.

Read the full Article as PDF:  SAML2_Web_SSO_Tutorial.pdf

Download the source files: SAML_SSO.zip (18 KB)