Coders are notoriously lazy - seriously. We’d normally rather find someone else that has developed the same thing or something close to what we want, copy it, then hack it up to do what we want. Its normally not like the movie Swordfish where you see some guy dancing and singing in front of a multi-screened computer just thinking up code (although, if you catch us late at night, around 1-3AM, you might see such a sight). We typically think up a few lines of code, then look at the news and then go back to coding a few more lines. If we need a big block of code like a function, we’d rather go searching, sometimes for hours, for someone that’s already done it for us.
Search engines for coders have been non-existent until the last year or so and looking for code in normal search engines is like a trip to the dentist and talking to a lawyer all rolled into one. Krugle and a few other Vertical Code Search Engines came out very recently to help the lowly developer on their quest to borrow, learn, steal and use from others - that is, of course, the Open Source way…
But what makes Krugle stand out from the others? How do they rank their searches? How many projects do they actually have? Luckily, I was able to talk to the core AJAX developer and the company founder via email and both had several interesting things to say. I got so comfortable talking with these guys that if I actually blogged full time, I might have asked to fly in to see them in action…
In The Beginning
Ken Krugler was talking with a friend at a start-up and a question came up: “Take a guess at what the most powerful development tool we use is”. Being a developer himself, Ken responded with Google and, unfortunately, he was correct. On the inside of Krugle, this is known as the “Google habit”.
Ken stated: “The unfortunate aspect, to me, was that Google didn’t do a very good job of being a search tool for programmers. I was frustrated by a constant feeling that I wasn’t finding what I needed, that it was taking too long to dig through irrelevant results, and that I needed a different way of searching for information.
The “Google habit” is the instinctive use of Google for all searches, even though there are much better vertical search solutions. And to a lesser extent an expectation that if Google doesn’t find it, then it must not exist. I think all vertical search plays have to deal with this same problem. “
After that conversation, some attempts where made to start-up Krugle, but it didn’t happen for awhile. He even required a stern lecture from a serial inventor that was a friend. After that, he got moving and met with Steve Larsen, Bill Daniher and the VC universe all businesses attempt to break into. The original version of Krugle was Keyword based but Ken and the developers quickly determined that it was not in the best interest to do so. After he realized that developers would want to search the entire document including functions, comments and other tidbits of information, things really started heating up. After a change here and there and a few financing rounds, Krugle officially went live, after a decent Beta phase, at 5:31PM on the 14th of June, 2006.
Krugle?
One has to wonder why the name Krugle was chosen. Is it a neat marketing ploy, just a place holder from someone’s name or what? Well, I’ve got the scoop directly from Ken Krugler…
“The name was initially just a placeholder - I called it “Krugle®” as a joke, and then when I needed a name to register the company (around March 2005) I figured I’d just go ahead and use it. But for better or worse it wasn’t a totally sucky name, so it wound up hanging around through the series-A funding, and that made it semi-official.“
Even the Krugle staff get involved in the Krugle name once in awhile… Oh, and there is a rumor going around that even Google got upset about the likeness of the name and sent Krugle a letter….
Krugle Staff
When Google released their Code Search Engine, myself and many others released reviews on Google, Krugle and Koders. Even though I didn’t come out and say it since the article was supposed to be unbiased, I preferred Krugle to Koders and Google Code Search (GCS). After I released my review on Krugle and the others, Chris Burmester, the lead AJAX developer, was kind enough to talk about what Krugle could and could possibly do. He later came back with an update:
-
Chris Burmester Says:
October 29th, 2006 at 2:12 am e
Justin - closing the feedback loop - we just pushed another release live at Krugle (www.krugle.com) and you’ll (hopefully) be happy to see line numbers on code files, as you suggested, that you can toggle on and off.
Plus, I talked with our lead code search API developer (Grant Glouser) about the white space in the code summary and he agreed that it really didn’t make any sense so - presto - gone.
You’ll also find the ability to create Krugle “Codespace†Links - URLs that capture the entire state of your search - complete with all the search results you have open, the results of your latest search, and the state of the code directory tree. Find some good stuff, create a Codespace URL, and share it with your friends/co-workers.
You can also use “lang:†as well as “language:†shortcuts in your searches. Plus, you can pretty much right-click or cntr-click on anything in the interface (links, tabs, widgets, code tree) to grab a full fledged URL to the item or action to share with others.
Please give it a try and let us know what you think. Lots of other improvements too.
We do “dot†releases about every two weeks and major releases about every six weeks - all heavily informed by our user feedback and what we find out here in the blogosphere. So please keep the feedback comin’. USE it (please) and tell us how to keep making it better.
Regards -
Let me tell you, as a lowly Blogger, I couldn’t believe that these updates where implemented. I’m sure others asked about certain features, but just the thought of a company making changes that I can actually see after a review is completely awesome. It’s obviously in Krugle’s best interest to be proactive to stay ahead of the Google Mania…
In trading emails with Chris, I found he’s a completely open and honest guy and one of the most friendly people I’ve met online, bar none. He was also the guy that introduced me to Ken Krugler.
 |
Profile: Ken Krugler - CTO Krugle.com Education: MIT Ken’s average day in his own words: “Whatever is the most important thing to get done next :) On a good day it’s a mix of planning, reviewing specs, and (if I’m really lucky) development. On a bad day it’s answering emails, filing Jira issues and figuring out what isn’t going to happen as soon as I’d like.” He’s also known to frequent the Krugle Blog |
 |
Profile: Chris Burmester - Senior AJAX Developer Krugle.com Education: UC Berkeley Chris is very proactive in finding Blog posts about Krugle and occasionally posts to the Krugle Blog. When he isn’t running around the Blogosphere, he does the majority of AJAX and other development work for the Krugle front end/UI and middle ware. He’s also so dedicated to Krugle that he talked with me while on Vacation. To his lovely Wife, I’m sorry for the lost time during vacation and I thank you for not killing the guy because of it (my wife would have smashed my laptop - seriously)… |
The Code Search
If you’re into soft green glow of the Krugle website (you thought I was gonna say Slashdot, right?), AJAX features out the wazoo as well as technical notes from other users, Krugle is for you.
The code search has more features than Google’s Code Search, happens to have better search results than Koders and has recently added more languages to their search ability than I care to list (seriously, go to their website and click on the language drop down.)
I did want to mention something though. I mentioned that Krugle had advertisements (which I said got in the way) and such and the interface was a little odd to learn. I still stand by the interface remark (Check out the Demo Webcast on how to use it.), but oddly, I was completely wrong about Advertisements. Krugle is quoted as saying they’ll make money through advertising, but they really don’t get in the way of what you’re looking at. They’re so subtle that you almost have to look for them.
The User Interface (UI) is something that Krugle constantly updates and tries to make better. You can search code, projects, books and the all the white papers you can get your hands on. I’ve never seen features like this built into one UI.
Feedback
Krugle craves feedback so much that its embedded into the website with AJAX. Find something that annoys you or a bug? Click the feedback link and it pops out a window over the current screen. After submitting the feedback, you’re right where you left off, no hitting the back button, no lost results, etc. Its obvious that Krugle listens to the feedback submitted on their site and from the community blogging about them and I applaud them for this.
When writing this story, I found an interesting bug caused by Krugle running inside the Google Cache engine. Pretty quickly, Ken Krugler wrote back after seeing my report in the company wide circulation of the Bug and Feature request email and thought it was pretty funny as well. Chris had talked about this internal email, but to see someone higher up on the Corporate food chain reading it, that says something. On top of this, Chris stated on numerous occasions that Krugle has the best “attitude” in handling feedback directly from visitors.
Now, to be fair and just, Krugle also uses a third party to help with their developer surveys. They have a pretty close relationship with Evans Data Corporation to assist with digging through all of the feedback. I’ve even seen developers online that have gotten a lot of the free things they offer. (How often do you see something listed as “you can win this” but you never know if someone actually won or if it was just a marketing ploy?)
When Ken replied to my bug report, he cut and pasted a few entries of feature requests from other people. One that caught my eye was “Please support MIL-STD-1589C JOVIAL (J73) which is hugely important in the military industrial people-killing business“… So, they apparently get all kinds of neat feedback.
Now, you may wonder why I’m really talking about feedback. Sure companies ask for it, but Krugle asks for it, then asks for it again, then bribes you with freebies then asks again… Rinse and repeat as many times as necessary… They are pretty damn serious about feedback - note the following items, for which I’m directly responsible for (or at least I’d like to think so… If I’m not, just let me live in my own little world ;-):
- Removal of blank lines in search results
- instead of the drop-down for choosing a language or typing out “language”, type lang=blah like GCS
- Line numbers
Krugle is so set on Feedback that they changed their UI four times during the Alpha and Beta phases. Oh, and because of those changes, Krugle is not an Eclipse RCP-based Java app…
If you are one of those people that just has to talk to THE people in charge so you can give feedback directly to the actual developers of Krugle, they often hang out at conferences like ApacheCon, OOPSLA, and AJAXWorld. You can even submit feedback and feature requests via their Forum.
The Technology Behind Krugle
The Krugle website and staff use a wide range of technology to keep everything functional on a daily basis.
The websites, databases, etc are run with a lot of Open Source packages like Nutch, Hadoop, Lucene, Solr, ANTLR, Apache, Linux, Perl, Python, Lighttpd, MySQL and others.
All of the servers run Linux, currently in the form of Fedora Core. The front end web servers run Apache, the back end servers use Jetty/Resin for Java servlets, and the crawlers use Nutch/Hadoop. When asked about the hardware, Ken replied that the “hardware we buy commodity x86 servers, with about five different configurations.”
When asked why they picked Fedora Core, the response was that it’s well supported and runs across all the different server configurations that Krugle deploys. Since they use Java/Python/Perl/Unix, their entire system is pretty platform-neutral. This configuration also lets their developers work on their favorite laptop. This translates into mostly Macs, a few PCs (aka: Windows), and one or two Ubuntu sightings have been reported.
On the website side, besides the behind the scenes stuff, Krugle uses a heck of a lot of AJAX. Chris Burmester and his team really do a great job of making it all work together. I would personally go crazy having to make sure it all played well together. One of the features I mentioned above, and that he’s proud of, is the saved sessions via deep linked URLs. Basically, when you find a spot that is exactly what you want, but you need to email it to someone else, click on the Create Link and their database will store your entire session - all tabs, phrases, etc for later use. If you use this hand in hand with the line numbers, you can send a link to someone and they’ll know exactly what line numbers you’re talking about. Chris says that, at the moment, the team sees the links being held as long as they possibly can - so dig out an email from two years ago and it’ll still work…
While I’d love to dig more into the technology behind the company, they currently don’t want to give the recipe away for the secret sauce…
The Projects
Google is notoriously rated #1 in the search engine circle, but when you drill down into “vertical searches”, for coding, Krugle is setting themselves up as the leader. Krugle currently has 50,000 projects indexed from Sourceforge, Freshmeat, Apache and others. Notice that it says Projects, not files. To get down to the nitty gritty, they have approximately 20 million source files, which is about 1 billion lines of code. Google doesn’t publish how many projects they index, but I’m going to guess its a larger amount because of the way they index.
Krugle currently does not get into the whole issue of crawling over each and every release of every single project. They rightly consider that a waste of time. They also are pretty careful about what they do crawl over and attempt to not keep anything that is proprietary in the system. While Google may attempt the same thing, since they crawl a different way, its a bit harder. For instance, as seen on Google:
* This file contains proprietary and confidential information and
* remains the unpublished property of Open Market, Inc. Use,
Whereas Krugle has proactively removed some proprietary code from Motorola and others that ended up inside their results a few weeks ago. While I think that if code is published online, its fair game, I can understand why they would want to proactively protect themselves from any type of issues that may pop up.
There are also projects inside the company that will interest the development world. They’re currently working on an Enterprise version of their search engine, a “search only my internal projects” type of thing as well as a developer IDE for use inside Eclipse and Visual Studio. While they won’t say when either will be made public, several people inside the company said the projects are being worked on rigorously.
So, after all of this, you find that your project is not in Krugle and you want to submit it - where do you go? At the moment, submit Feedback and there are a few Submit categories. They’re working on upgrading this, but are testing various ways to do it. They even have a librarian to look over everything to ensure that all the information like license, project name, etc is correct.
In The End
You can see that I’m pretty excited about Krugle. After spending years digging through regular search engine results, this is one tool that I use on a regular basis now. In fact, I got so excited about Krugle, I spent all this time away from the real world emailing Chris & Ken (btw - Thank you both) while getting dirty looks from my wife (everyone’s wife complains that their husband spends too much time with the computer, right?). To my neighbors, I’d apologize about all the music at 2 in the morning while I was writing, but I don’t like ya anyway.
