The Inevitable Convergence of Middleware into the Network
Today in New York our CEO Craig Betts co-keynoted the High Performance on Wall Street conference with Andy Bechtolsheim and a representative of Barclays Capital. The theme was something we talk about a lot at Solace: the convergence of middleware and the network. The need to effectively route an increasingly massive volume of data at high speed is defining competitive advantage in more and more situations. The only way we’ll ever address this trend is by making the intelligent routing of messages a robust and readily-accessible service of the network, just like the routing of IP packets. We can only shove so many servers into the increasing number of holes in the dike before the water pours through.
Craig talked about the 4 key business drivers of IT innovation today (the need for speed, the need to save, the desire to be more environmentally sensitive and the need to stay productive through M&A activity) and how they all lead down this path of convergence. Mr. Bechtolsheim dove a little deeper into the specifics of network architecture and operations to illustrate the point that raw computing power isn’t the way to crack this nut. Finally, Barclays Capitals’ head of middleware talked about the business value of consistent, predictable performance which drove them to select a (ok, our…) hardware-based middleware platform.
From business drivers, to nuts and bolts, to the bottom line, the presentation made it clear that the migration from old-fashioned software-based infrastructure to hardware is a fundamental corner we must turn.
Ushering in the Unified Messaging Platform
History Has Left Us with a Potpourri of Messaging
From the earliest days of distributed computing, the idea that steps in a workflow use a task queue of some kind to pass off information has been a central tenant of information design. Over time, major buckets have emerged, each on their own timelines, and each with a different technology that became the de facto standard.
- Message queuing — Dominated by MQ Series (er… WebSphere MQ), but there have been many imitators along the way: DEC Message Queue, Oracle AQ, Microsoft MSMQ, etc.
- Publish subscribe — Pioneered by Tibco with Rendezvous and its precursors, but also used in SmartSockets, 29West, RTI and others.
- Ultra-low latency — Primarily the same names as publish subscribe, but in a different configuration
- JMS — JMS was the first attempt at an open messaging standard with interoperability defined only at the API level.
- AMQP — A working group that’s defining a standard wire protocol for publish subscribe and message queuing. This is the reverse of JMS as it would focus on multi-vendor interop, but does not (currently) plan to enforce a single API.
- Wide area network — For low volumes, you can use any guaranteed or queue-based messaging to reliably get information across a WAN. For higher volume, most users have custom architectures that batch messages into larger groups and send as either messages or FTP files for greater efficiency.
- Other messaging — You will find a few other messaging types, such as computer to human messaging protocols like XMPP, which was initially developed for use with internet chat programs but has been extended to include additional functionality.
The Messaging World Has Fundamentally Changed
Most enterprises have ended up with a mishmash of all of these types of messaging–often five, ten or even more solutions from a handful of different vendors. This isn’t usually the result of poor decision making, but of the fact that at the time each one was deployed, it was the fastest, easiest or most reliable way to meet a given requirement. But hardware fundamentally changes what’s possible in middleware.
Today’s limits are either limitations of the interplay between operating systems and software, or the limits of supporting multiple general purpose hardware servers. For any one of these kinds of messaging, hardware leaves these limits in the dust and blows the doors off what’s been possible with software-based solutions:
- Publish subscribe fanout of more than 10 million messages a second (basically saturating a 10 GigE link)
- Fully failsafe message queuing at 130,000 messages per second
- Ultra-low latency messaging with 25 microseconds of latency and very low jitter
- WAN streaming that exhibits orders of magnitude more throughput than software over the same network
Each one of these represents out-of-the-box performance that surpasses what world-class architects would not be able to get out of software. But what if you could consolidate all of these parallel products into one footprint as well?
All Messaging Under One Roof
The real power of the Unified Messaging Platform is not that it can run circles around software in each of these areas, but that it lets you comfortably use the same equipment for your front, middle and back office capabilities without worrying about one application impacting the performance or stability of another. You don’t build a new IP network for each application, do you? Similarly, it’s time to stop building, scaling and designing redundancy for so many discrete middleware systems.
If you could (over time) replace your MQ and JMS infrastructure, as well as your market data delivery and specialty algo trading infrastructure, in favor of a single network layer that could do all of these better, faster and easier, why wouldn’t you? The ROI is a no brainer. The business gets worry-free headroom and the ability to scale to tens of millions of messages per second. The business gets competitive edge in operational improvements and faster trading.
It’s a game-changing value prop and it was unthinkable just a few years ago. If you ask the really talented architects and infrastructure designers on the street today, they are more likely to call it inevitable than unthinkable.
The evolution and future of Ethernet — is it the perfect predator?
Sharks are so deadly and efficient that they’ve been called “the perfect predator.” That’s why I find it fascinating that they’ve been the top predator in the oceans for literally hundreds of millions of years, and haven’t changed much since the extinction of the dinosaurs. Why not? They didn’t need to! Their basic design was so good that it only needed tweaking to survive while other species in water and on land died out (sorry, dinosaurs) or evolved so much you wouldn’t recognize them from their ancestors. (sorry, creationists)
Sound like any networking technologies you know? While I hesitate to admit this, I’ve been using Ethernet since before some of you were born. Ethernet may not be perfect, but it’s very much like the shark in that aside from improvement over time (higher bandwidth, new kinds of wires) it’s a lot like its ancestors. In fact, I have some “museum piece” computers at home running 10 MbitE that are talking to my 1 GigE hosts.
Read more
Honey, 10GigE shrunk the datacenter!
With all the talk about speed and throughput, I think a lot of people look past the most concrete and immediate benefit of 10GigE: more compact, less costly data centers.
Simply put, if each wire can carry 10x the data, you need 1/10 as many wires to connect all your servers and storage devices. The ratio isn’t precise, and it doesn’t extend to network equipment where devices have 24, 48 or more ports, but clearly when you have fewer fatter pipes you can migrate to an infrastructure composed of fewer, but more powerful, routers and switches.
At least in the area of messaging, you can compound this consolidation by replacing racks of servers running messaging software with hardware-based messaging appliances like Solace’s. In terms of workload, each hardware-based router can replace between 10 and 20 middleware servers while providing better performance and predictability. It’s important to note that all those servers running messaging software won’t benefit very much from 10GigE, something I’ll touch on below and covered in great detail yesterday.
Companies are pushing the envelope of data center density in the quest to keep costs in check while supporting increasingly insane processing and throughput requirements. 10GigE and hardware-based messaging promise to free up countless racks of space and save millions of dollars. Here’s some of the reasons why, and some of the factors that need to be considered when shrinking the datacenter.
Read more
10GigE; What will fill the pipes?
It seems to me that every time a new generation of high-capacity networking technology makes its debut, there’s an accompanying list of applications ready to use up all that bandwidth. It’s taken for granted that applications will always find new and creative ways to use up all the available bandwidth. I’ve been disappointed, however, with the list of interesting use cases for 10GigE. It just seems like many are the same old bunch from when 1GigE launched — social networking, video, etc.
Does that mean we’re on the verge of having more bandwidth than we need? Hardly. I believe the new applications that will fill the 10GigE pipes are not entirely new at all – they are mashups of applications, services and trends that already exist, combined in ways that will chew through bandwidth like nothing we’ve ever seen.
Read more
Will you and 10GigE live happily ever after?
Over the past few decades we’ve seen network technology evolve from 10 Mbit Ethernet to 10 Gbit, with 40 GigE on the horizon and 100 GigE already being discussed. Is the network outpacing the capabilities of the computer? In many ways, yes.
One of the dirty little secrets of today’s operating systems is that even in multi-CPU hosts all the interrupts from the network have to be processed on a single CPU. This means a single 1 GigE card can overwhelm even the fastest commercial processors available today. So what will 10x the throughput do for your servers if the CPU/OS is the bottleneck? In many cases, the answer is not very much. I think many administrators and architects may be prematurely falling in love with 10GigE when it won’t really solve as many of their problems as they hope that it will.
Don’t get me wrong, there are lots of applications and environments out there that will benefit greatly from 10GigE. And there’s a whole slew of cool new applications and services and capabilities that it will make possible. I just think it’s important that people recognize that 10GigE isn’t a silver bullet that will open up a new world of performance in every situation.
Read more
You think you’ve got multicast storms now? Just wait for 10 GigE!
Back in the early days of market data systems, 10 MbE networks were the standard. Multicast from the publisher to the subscriber was a clever way to optimize bandwidth to trading desks, because there just wasn’t enough bandwidth to send the data repetitively to each trader. The emergence of 100 MB and 1 GigE networks resolved some of the bandwidth issues, but the delivery bottlenecks just shifted to the software middleware, so multicast lives on.
But multicast has a dark side. If you want to see any Wall Street or internet infrastructure architect get worked up, ask them about multicast storms. Multicast storms happen when application participants request retransmits of information they have missed in the multicast stream. There are two common causes of multicast storms:
- Applications that fall behind in their rate of consumption of messages
- Network speed mismatches in the underlying network.
As market data rates accelerate and trade volumes go through the roof, many people are counting on 10 GigE to bail them out just as things get really hairy. Unfortunately, the migration to 10 GigE won’t be immediate, and it won’t be universal, so the emergence of 10 GigE will actually exacerbate the second common cause of multicast storms. In context of those increasing market data rates and trade volumes, some predict a “perfect storm” in the world of trading systems.
Read more
10GigE enables world’s fastest real world market data benchmarks
In Nov 2008, we released a set of benchmarks for ultra-low latency market data distribution. The tests, run on an all-hardware architecture consisting of 1 GigE technology from Solace, Arista and NetEffect, showed the fastest real-world messaging benchmark performance ever seen — until today.
In conjunction with the launch of the new 10 GigE version of our Network Acceleration Blade, we repeated these tests in a 10 GigE environment. In addition to a 10 GigE-equipped Solace 3260 Content Router we used NetEffect 10 GigE adapters to perform TCP offload in the clients and servers, with an Arista 10 GigE cut-through switch as the layer 2 switch.
To provide an apples-to-apples comparison with the previous benchmarks, we reran the test that simulated all US equities traffic (500K msgs/sec) and the test that simulated peak OPRA market data (1 million msgs/sec.) Then we added to the mix a 2M msgs/sec test to show how performance changes as data rates continue to go up.
Once again, the performance was superb, and the latency distribution nice and tight. The system demonstrated latency 20-25% lower than the previous 1 GigE numbers and improve upon our already best-in-class performance. Here’s a summary of the results:
Brace yourself: It’s 10 GigE week!
Each summer, the Discovery Channel dedicates a week of programming to Sharks, with the now infamous “Shark Week” celebrating all things Shark.
Here at Solace, we decided that this week will be “10 GigE Week,” celebrating the emergence of 10 Gigabit Ethernet networks. We’ve blogged about 10 GigE before, but never to the level of depth we will this week.
So if you’re into messaging or middleware, and you’re adopting or planning to adopt 10 GigE, watch this space. You’ll see our architects get a little closer than is safe, or maybe see 10 GigE devour an unsuspecting Infiniband network. The only thing you know for sure about 10 GigE Week is that, just like Shark Week, you don’t want to miss it…check out all the 10 GigE posts (so far!) right now.
I see latency…it’s everywhere.
A recent article in the Wall Street & Technology journal summarized the consensus of experts who attended a conference dealing with low latency on Wall Street. The main premise was this: “The weakest link in the low-latency value chain is older software or poorly written code, not market data feeds, lack of ultra-fast processors or older networks.”
I’ve experienced this myself in previous jobs, and it’s not inaccurate. The problem is when people run with that and think they should fix the software side of the equation with a blind eye to a full end-to-end solution that encompasses all technologies. It’s a dangerous leap of logic exemplified by the article’s subtitle: “Panel at Accelerating Wall Street conference says improving older or poorly written code has greater potential to lower latency than new hardware.”
Don’t get me wrong – optimizing your applications is a critical piece of the puzzle, and speeding up your app might be the single biggest performance gain you can realize. I also agree that throwing new faster Intel boxes at the problem and running the same poor codebase on faster hardware is a poor choice. But as I said in the title, latency is everywhere. If you don’t continuously explore the game-changing potential of new technologies (such as 10GigE, cut-through switches, TCP offload engines and hardware-based messaging, in-memory databases, caching technology) you will forever lag behind competitors who take a holistic approach to latency and invest the time and energy to incorporate important new technologies into their system.
In today’s day and age, you have to explore every piece of technology to achieve the lowest possible latency. There’s just too many exciting developments happening in the areas of networking, middleware and new disk technologies, etc.