LNAV: The Log File Navigator

Today I wanted to throw a shameless plug and highlight one of the side-projects from one of our very own engineers, Tim Stack.

For developers, ops, and those who frequently live in the terminal, lnav is log file viewer that can help you analyze and navigate through your logs. Simply point lnav at a file or a directory and it will decompress the files, detect their format, and index any log messages contained within. The log messages from all files are then displayed in a single view ordered by time and several hotkeys are available for jumping to messages based on their time or log level. Semantic information is highlighted and filters may be applied to help focus your investigation. Additional features include a pretty-printing structured data, like XML or JSON, a SQL interface for querying logs and more. To check out the complete list of features and download for Linux or Mac OS X, head on over to http://lnav.org/

Here are a couple of screenshots: Continue reading “LNAV: The Log File Navigator”

SnapLogic Is Hiring – Engineering, PM, DevOps, QA & More

SnapLogic-is-hiringSnapLogic is currently on the hunt for engineers, marketers, QA specialists and more. We’re building the team in our San Mateo, CA headquarters and continuing to expand our Boulder, CO office and seeking out talent all around the country for both in-office and remote positions. Are you currently in Boston or New York? We have offices there too and we are expanding internationally in 2016. Some of the teams that are currently hiring include:

  • Product Management
  • Marketing
  • Sales Development
  • Human Resources

And in engineering we are hiring for many important roles, including:

  • Big Data Developer
  • DevOps Engineer
  • Java Application Engineers
  • Tech Support Engineer
  • Solutions Engineer

You can check out all of our open positions here.

As more and more enterprise organizations recognize the importance of data and application integration to cloud and big data success, we have grown significantly in the past year with some important hires and the opening of additional offices. Learn more about the perks and benefits of working at SnapLogic.

A little bit more about SnapLogic and the company’s history:

SnapLogic is the industry’s first unified data and application integration platform as a service (iPaaS). Our hybrid cloud architecture is powered by 300+ Snaps, which are pre-built integration components that simplify and automate complex enterprise integration patterns and processes. Funded by leading venture investors, including Andreessen Horowitz and Ignition Partners, and co-founded by Gaurav Dhillon, the SnapLogic Elastic Integration Platform enables leading enterprises to connect faster and gain a better return on their cloud application and big data investments.

Engineering – What We Value at SnapLogic

At SnapLogic we have a high quality bar for the software we create. Our products must be top-notch to compete in the enterprise integration space so we can’t compromise when it comes to engineering best-practices. Below are a few of the things we do to ensure our bar stays high. Nothing here is too revolutionary, just good common sense.

Code Revision System
Like many other companies we’ve adopted git to track changes in our code. To avoid the operational overhead of maintaining our own central repo we use GitHub. The service is fantastic and we can sleep at night knowing our code is safe and secure in the cloud.

Peer Code Reviews
Not a single line of code is checked in without at least two engineers looking at it. This greatly reduced the number of defects introduced into the code and is a natural way to have engineers cross-train on different parts of the code base. As human beings we all make mistakes, but with the help of our colleagues we have a better chance of catching them. For more insight into this aspect of how we operate, SmartBear Software has an excellent whitepaper discussing the merits of good code reviews.

Coding Style Guide
Code artifacts are the main deliverables of our daily activities. Very much like works of art, we want our artifacts not only to be fully functional, but to also be beautiful objects of which we can be proud. Beyond beauty there are other benefits of a consistent code base:

  • Well-written and well-documented code is easier to upgrade and maintain
  • Bugs are easier to find and fix, earlier in the development lifecycle
  • More re-use of existing artifacts

Additionally, a consistent coding style ensures that people will be most able to work in different areas of the products without having to spend an inordinate amount of time having to familiarize themselves with unfamiliar idioms and constructs. We’ve primarily settled on PEP-8 for Python, Oracle’s style guide for Java, and Google’s style guide for JavaScript.

Continuous Integration Environment
Every engineer’s best friend is a jolly fellow named HudsonJenkins who answers the age-old question “will this code break the build?” Jenkins happily checks out your code, builds it, runs tests, pushes artifacts to Artifactory or Nexus repos, and finally deploys it. No more surprises at the end of the release cycle!

Hermetic Builds
Every binary that is compiled in our continuous integration environment is deterministic, meaning we can build a bit-wise identical copy of any artifact from source. This requires a static build-tools chain with versioned OS, compiler (with flags), and dependencies (.jar, .so, .egg, etc). Hermetic builds guarantee we can re-create any existing artifact just from a git commit hash. Gone are the days when the binary running in production was built on some random engineers laptop.

Automated Testing
Unit, smoke, regression, performance, lemon, negative, positive, stress, load, penetration, and error tests. What do they all have in common? A machine can do them! Manual testing of software is time consuming and inconsistent. Save your QA’s precious time by having them tests those tricky, non-deterministic or UX cases that just can’t be done by a machine. Don’t make your QA just press buttons. They’ll thank you for it and your users will get a better product.

Code Coverage Metrics
Peter Drucker, an often quoted management consultant famously said “If you can’t measure it, you can’t improve it.” This quote aptly applies to code coverage; if you don’t measure how good your unit tests are, you can’t have any real confidence in the quality of your code. There are plenty of fantastic tools out there that report coverage — Eclipse and IntelliJ both have coverage baked right into the IDE. This helps engineers write comprehensive tests up-front and tools like Sonar can tell you if those tests add value.

Repeatable Release Process
Each push to production should be audited, repeatable, and easy. We currently use a one-button push via Jenkins that reliably pushes new binaries to our production machines via puppet. It then systematically restarts each service and sends the push event to Graphite. This process takes one mouse click and any of our release engineers can do it. No manual steps, no cut-n-paste deploy commands, and no bespoke push scripts. This makes rollbacks simple too!

“Leave this world a little better than you found it.”
Last but not least, we operate under the philosophy made famous by Lord Baden Powell, founder of the Boy Scouts. Whatever code we touch we strive to leave it in better shape than we found it.

If you too value the above, get in touch. We’re always looking for like-minded engineers to join the team! You can get in touch at SnapLogic.com/jobs.