Maker Days 2014 Q2 Hacks
“SpeedLedger e-bokföring is somewhat our flag ship of products. The product relies heavily on the connection to internet banks. All accounting is based upon bank transactions and future payments. Today we have two ways of getting hold of bank data and we host our product on two sites. The first one is hosted within an internet bank with direct access to the bank internal APIs. The second one we host ourselves, without access to an API we have come up with another technical solution. Obviously, we would prefer using internet bank APIs, unfortunately they simply do not exists or offer everything we need.
Our current technical solution is based on a virtual printer. The users log into their internet banks with instructions on how to transfer the content to us. They print the pages we need in order to build up smart accounting. Our printer converts the print stream to PDF format and sends it over the wire to our back end. And here is a real challenge. Parsing the content into Java objects.
Our project addresses two problems with the current solution:
- The user must know how to navigate to the pages we are interested in
- Parsing PDF data is really hard and error prone due to OS, browser and user behavior
Firstly we made sure that the user’s internet bank is reached by clicking a button within our app. A modal dialog containing an iframe with the internet bank inside is opened. The url of the iframe points to the login page, and the user logs in. Thereafter a Chrome plugin kicks in and navigates the internet bank pages for the user and sends the content to our back end. The plugin can be ran in two modes in this proof-of-concept. Either showing the user all navigation needed in internet bank or only the login page and thereafter a “progress bar”.
So we have come up with…
- A Chrome content script plugin taking care of navigation and posting html to server.
- Minor changes to SpeedLedger e-bokföring; a hyperbank button and a dialog containing an iframe.
- A Groovy back end receiving bank page html. Parses the html and puts in on a JMS queue for processing.
One Click Template
Our support team spend a lot of time tuning templates in a product called SpeedFeed. The template GUI enables the users to set keyword elements so that invoices can be parsed and passed into our system. This work is tedious and needs tweaking for many customers.
Most of the creation of the template can be automated by looking after keywords on the invoice. For instance, the invoice number is often located near a label with the text “invoice number” or “invoice no”. By looking at the format of the data, further accuracy could be added, for instance just by looking at these strings “SE123456789101” “2014-01-27” you could guess that one of them is a vat registration code and one is a date.
So the result is a button triggering the search algorithm that finds likely candidates for each field.
One login to rule them all
Allow employees to access internal services with a single login so that they don’t need to remember several different accounts or share a common password. It’s also intended to ease development of new internal services since authorization is a problem that is already solved.
A reverse-proxy that requires that the user authenticates with his/hers Google login before any request is sent through the proxy. The server is written in Scala with Akka and Spray.
The proxy does not rely on static configuration files for proxies but instead have a REST API to add new proxies at runtime. Currently the dynamic configuration is lost on server exit, the intent is to use a simple database to store the configuration as well as user sessions.
The API can be used to build a dashboard to view which services that are available behind the proxy, this should be limited by setting some metadata on the proxy configurations. A simple configuration UI can also be built.
GET, DELETE proxy-api.domain.tld/proxies/fancy-service.domain.tld/path
"value": "Basic XYZ"
GET, DELETE proxy-api.domain.tld/proxies/fancy-service.domain.tld/path
"value": "Basic XYZ"
When a user connects to the proxy it checks if it have a session cookie that is valid. If it is missing or not valid a redirect is made to a predefined login domain that may hold a session cookie.
If the login domain have the cookie then a request is made back to the original domain with the value of the session cookie. The session cookie is then set on the original domain and a redirect is made to the original URL.
If there is no cookie or it’s invalid then the user will be redirected to Google’s login. When the user have completed the login and we receive the callback we check that we got back a valid token and that the users email matches a regexp (for example “@example.com$”). We then grab the full name of the user and creates a session and sets the cookie on the login domain before making a redirect to the original URL.
“Who is who” – Next Generation
Our idea is to take the “Who-is-who”-page on our intranet to the next level.
The existing version is very popular and helps all of us putting names to faces and faces to names. A new version of Who-is-who has the potential to be an important tool in building the SpeedLedger corporate culture.
Built as a web application with Google single sign on, Who-is-who NG can seamlessly be integrated into the existing family of internal SpeedLedger applications while maintaining a very high level of user authentication.
Zero Install Screen Sharing
Allow our support staff to see the screen of the customer they are talking to, without requiring any changes to the customer’s computer.
After listening in on a few support calls we realized that a lot of frustration, for the support staff and customers alike, was caused by the fact that they couldn’t both see the screen they were talking about. Therefore we wanted to give the support staff the ability to view the customer’s screen.
There are a large number of screen sharing solutions on the market. However, all of them (that we know about) require something to be installed on the customer’s computer. It might be a small client software, or at least Java or Flash. We wanted to create something which did not require anything at all to be installed on the client.
We chose to base the solution on a library called html2canvas which renders the current DOM on a canvas. From this canvas we can then get an image which we could post to the server. The server saved the image on disk and an administrative GUI could then display them to the support staff.
We also created a “live” screen sharing mode where the client continuously sent updates to the server. However, an image was captured and sent only if something had changed in the DOM and at most every 3 seconds (could be configured). This was bound to key sequence s s.
The images are sent as base64 encoded PNG images.The client also attaches some basic information about the logged in customer; org number, org name, and user name.
The server we post to must be the same as where the page is loaded from to avoid cross-domain concerns.
Noteworthy is also that the problem of how to trigger the capturing is quite hard. A special menu alternative or button somewhere would probably not work very well, since the customer might be stuck in a modal dialog (that might be why they called in the first place). The key sequence solution we chose solves this, but has problems of its own; it doesn’t work if the customer has focus in a text field, and some users might have their browsers configured to start searching on the current page when a letter is pressed.
On the server side, we created a Servlet which took the uploaded image, decoded it and saved it as an image file. The file got the current timestamp as name. It also took the other information sent from the client, plus the value of the User-Agent header, and saved it in a metadata file. Both the images and the metadata were saved in a directory with the org number as name.
We created a GUI for the support staff where they could see a list of which organizations that images had been submitted. When a new image is submitted, the page would automatically update to reflect that.
They could click on one of the organizations and see the latest image submitted for that org, plus the metadata for that org/image. When a new image is submitted, the page would automatically reload to show the latest image.
The GUI was implemented as a Angular.js application backed by a simple Node.js server.
Result and Considerations
We got a solution which works quite well. However, there are a number of things that we would have to change or take another look at.
- A general cleanup and hardening of the code.
- Consider other ways to trigger the capturing.
- Make the “live” capturing adjust the “framerate” based on the performance of the customer’s computer and the network speed.
- Decide whether we should allow screenshots to be sent for customers who are not logged in.
- Make it work behind our proxy (shouldn’t be too hard).
- Find a way to make it work behind our partner banks proxies (might be harder or even impossible).