Hardware-based middleware gets some love in the Journal
The migration of middleware from software to hardware continues to gather momentum and market recognition, as evidenced today by an article in the Wall Street Journal about how financial services firms are implementing solutions that leverage performance-oriented hardware historically used in gaming and science to squeeze latency out of their trading systems.
Trading firms are transitioning from CPUs to field-programmable gate arrays, or FPGAs, graphics processing units, or GPUs, and cell processors. Such chips cut out the need for sending information across the operating system, and have the ability to run numerous processes in parallel, or simultaneously. CPUs actually have faster raw clockspeeds but can primarily only do one thing at a time, which ultimately slows things down.
“You’re basically sacrificing raw clockspeed for parallelism,” said Mike Dunne, chief technology officer at Activ Financial, which uses FPGAs in providing market data. “That trade-off was worth making, even though the clock speed’s lower, because we take advantage of the parallelism, as well as the power and the fine-grain memory control. You can get a 10-times or even 24-times increase in your performance.”
Solace was mentioned in the article by recently announced customer Liquidnet.
“We were frankly pushing our legacy systems to their limits,” said Keith Piraino, head of enterprise architecture at Liquidnet, a block-trading venue that recently announced plans to transition from software to hardware-based middleware from Solace Systems.
“We evaluated 10 different vendors for a solution and learned during this process that things are moving more into hardware,” he added, citing lower latency, the consistency of that lower latency, and lower costs among the reasons why Liquidnet chose to undergo hardware acceleration.
It’s nice to see one of our customers, and our story, get some recognition in such a respected publication.
Back to the future: the return of appliance-based computing
There was a good article on the Enterprise Systems website this week about the increase in popularity of purpose-built appliances on many fronts across the datacenter.
The points it makes will be familiar to readers of this blog:
IT organizations like the model for the same performance and security reasons vendors do, but also because deployment is a breeze and the initial and on-going costs of appliances are considerably less than software-only solutions or SaaS-based solutions. With the appliance model, gone are additional hardware and software prerequisites because the black-box approach of hardware appliances bundles everything needed into an all-in-one, turnkey solution.
Also gone, and far more considerable, are the professional services costs of the integrator needed to handle the complicated task of installing and integrating new software into an existing environment. Finally, and although varying by solution area, the inherent focus of most appliances is to hide complexity from its end user and thereby simplify its installation, integration, and use. This simplification usually has the added benefit of reducing training time so administrators can be productive more quickly, further reducing the overall cost of the solution.
Hardware trading: The machines are taking over…
With each day that passes it becomes more obvious that hardware is replacing software in the critical path of high performance trading. It doesn’t take a genius to see what’s happening, as the evidence is everywhere:
- Feed handlers: All of the recent entrants into the feed handler market are using custom hardware.
- Messaging middleware: Appliances from companies like Solace are winning a larger footprint for market data delivery and back-office order routing.
- Algo engines: Leading algo trading engines are heavily leveraging FPGAs and GPUs to do ultra-fast simulations that accelerate trading decisions.
- TCP offload engines: TOEs are taking the place of software TCP stacks on servers.
- Trading apps: We’ve even seen a flurry of requests from high performance trading firms looking to execute pre-trade risk checking directly in network processors inline with the hardware messaging bus.
Want a truly green datacenter? Turn off the servers.
You may hate the hype about “Green IT” as much as I do, but the problem it addresses is very real so don’t expect the term to go away anytime soon. There are some interesting stats on how out of control datacenter sprawl is getting in this article in eWeek Europe.
In 2020, data centres worldwide will consume about 450 billion kWh and their CO2 emissions (at about 330 million tones) will be equal to those of Portugal, Switzerland, Greece and Sweden combined. Their electricity bills will amount to nearly $45 billion (£26bn).
The demand for computing is always growing, so the only practical way to get more green is to do more work per watt.
Working on basic sharing skills
In a blog post this week, Lori MacVittie talks about, among other things, the reluctance of different groups towards sharing infrastructure. From her post:
Some pieces of infrastructure – particularly those that are part of the network – are so critical (and so very underappreciated) – that they simply cannot be exposed to the kind of risk that comes from “sharing” resources in any model.
I can’t say her never ever kind of experience matches with what we have seen with hardware middleware. When it comes to sharing, our customers fall into two main camps:
- Those that re-buy gear for each major business unit because of concerns over traffic from one interfering with another. This is how they have always deployed software, and they’re not ready to do it differently in hardware. It has more to do with CYA than technology, which is consistent with Lori’s point.
- Those that are blending market data for many asset classes (across departments), and/or a combination of market data, order routing and risk management in the same infrastructure. Generally the motivation is reducing costs, but it can also reduce risks when the shared environment is both faster and more stable than the standalone software it replaces.
The Inevitable Convergence of Middleware into the Network
Today in New York our CEO Craig Betts co-keynoted the High Performance on Wall Street conference with Andy Bechtolsheim and a representative of Barclays Capital. The theme was something we talk about a lot at Solace: the convergence of middleware and the network. The need to effectively route an increasingly massive volume of data at high speed is defining competitive advantage in more and more situations. The only way we’ll ever address this trend is by making the intelligent routing of messages a robust and readily-accessible service of the network, just like the routing of IP packets. We can only shove so many servers into the increasing number of holes in the dike before the water pours through.
Craig talked about the 4 key business drivers of IT innovation today (the need for speed, the need to save, the desire to be more environmentally sensitive and the need to stay productive through M&A activity) and how they all lead down this path of convergence. Mr. Bechtolsheim dove a little deeper into the specifics of network architecture and operations to illustrate the point that raw computing power isn’t the way to crack this nut. Finally, Barclays Capitals’ head of middleware talked about the business value of consistent, predictable performance which drove them to select a (ok, our…) hardware-based middleware platform.
From business drivers, to nuts and bolts, to the bottom line, the presentation made it clear that the migration from old-fashioned software-based infrastructure to hardware is a fundamental corner we must turn.
Ushering in the Unified Messaging Platform
History Has Left Us with a Potpourri of Messaging
From the earliest days of distributed computing, the idea that steps in a workflow use a task queue of some kind to pass off information has been a central tenant of information design. Over time, major buckets have emerged, each on their own timelines, and each with a different technology that became the de facto standard.
- Message queuing — Dominated by MQ Series (er… WebSphere MQ), but there have been many imitators along the way: DEC Message Queue, Oracle AQ, Microsoft MSMQ, etc.
- Publish subscribe — Pioneered by Tibco with Rendezvous and its precursors, but also used in SmartSockets, 29West, RTI and others.
- Ultra-low latency — Primarily the same names as publish subscribe, but in a different configuration
- JMS — JMS was the first attempt at an open messaging standard with interoperability defined only at the API level.
- AMQP — A working group that’s defining a standard wire protocol for publish subscribe and message queuing. This is the reverse of JMS as it would focus on multi-vendor interop, but does not (currently) plan to enforce a single API.
- Wide area network — For low volumes, you can use any guaranteed or queue-based messaging to reliably get information across a WAN. For higher volume, most users have custom architectures that batch messages into larger groups and send as either messages or FTP files for greater efficiency.
- Other messaging — You will find a few other messaging types, such as computer to human messaging protocols like XMPP, which was initially developed for use with internet chat programs but has been extended to include additional functionality.
The Messaging World Has Fundamentally Changed
Most enterprises have ended up with a mishmash of all of these types of messaging–often five, ten or even more solutions from a handful of different vendors. This isn’t usually the result of poor decision making, but of the fact that at the time each one was deployed, it was the fastest, easiest or most reliable way to meet a given requirement. But hardware fundamentally changes what’s possible in middleware.
Today’s limits are either limitations of the interplay between operating systems and software, or the limits of supporting multiple general purpose hardware servers. For any one of these kinds of messaging, hardware leaves these limits in the dust and blows the doors off what’s been possible with software-based solutions:
- Publish subscribe fanout of more than 10 million messages a second (basically saturating a 10 GigE link)
- Fully failsafe message queuing at 130,000 messages per second
- Ultra-low latency messaging with 25 microseconds of latency and very low jitter
- WAN streaming that exhibits orders of magnitude more throughput than software over the same network
Each one of these represents out-of-the-box performance that surpasses what world-class architects would not be able to get out of software. But what if you could consolidate all of these parallel products into one footprint as well?
All Messaging Under One Roof
The real power of the Unified Messaging Platform is not that it can run circles around software in each of these areas, but that it lets you comfortably use the same equipment for your front, middle and back office capabilities without worrying about one application impacting the performance or stability of another. You don’t build a new IP network for each application, do you? Similarly, it’s time to stop building, scaling and designing redundancy for so many discrete middleware systems.
If you could (over time) replace your MQ and JMS infrastructure, as well as your market data delivery and specialty algo trading infrastructure, in favor of a single network layer that could do all of these better, faster and easier, why wouldn’t you? The ROI is a no brainer. The business gets worry-free headroom and the ability to scale to tens of millions of messages per second. The business gets competitive edge in operational improvements and faster trading.
It’s a game-changing value prop and it was unthinkable just a few years ago. If you ask the really talented architects and infrastructure designers on the street today, they are more likely to call it inevitable than unthinkable.
Will you and 10GigE live happily ever after?
Over the past few decades we’ve seen network technology evolve from 10 Mbit Ethernet to 10 Gbit, with 40 GigE on the horizon and 100 GigE already being discussed. Is the network outpacing the capabilities of the computer? In many ways, yes.
One of the dirty little secrets of today’s operating systems is that even in multi-CPU hosts all the interrupts from the network have to be processed on a single CPU. This means a single 1 GigE card can overwhelm even the fastest commercial processors available today. So what will 10x the throughput do for your servers if the CPU/OS is the bottleneck? In many cases, the answer is not very much. I think many administrators and architects may be prematurely falling in love with 10GigE when it won’t really solve as many of their problems as they hope that it will.
Don’t get me wrong, there are lots of applications and environments out there that will benefit greatly from 10GigE. And there’s a whole slew of cool new applications and services and capabilities that it will make possible. I just think it’s important that people recognize that 10GigE isn’t a silver bullet that will open up a new world of performance in every situation.
Read more
Liftoff for 10 GigE content networks
We kick off 10 GigE week with a product announcement. Today Solace announced a new 10 GigE Network Acceleration Blade (NAB), along with some new 10 GigE messaging benchmarks conducted with our partners Arista and NetEffect.
As a recap, the Solace 3260 content router is a high-speed content routing chassis that houses a different set of hardware blades depending on the type of messaging or middleware function your applications require. No matter what type of messaging you’re doing, every blade combination starts with a Network Acceleration Blade or NAB. The NAB is the shared I/O blade that performs a wide range of functions in hardware. Each NAB:
- terminates Ethernet, IP and TCP in hardware
terminates the messaging protocols in hardware- can compress WAN traffic in hardware
- efficiently manages publish/subscribe content fanout
- tracks detailed statistics about client and server connections, queue depths and message rates without negatively impacting performance
This new announcement is the third NAB (network blade) in the lineup, the first to support 10 GigE networks.
I see latency…it’s everywhere.
A recent article in the Wall Street & Technology journal summarized the consensus of experts who attended a conference dealing with low latency on Wall Street. The main premise was this: “The weakest link in the low-latency value chain is older software or poorly written code, not market data feeds, lack of ultra-fast processors or older networks.”
I’ve experienced this myself in previous jobs, and it’s not inaccurate. The problem is when people run with that and think they should fix the software side of the equation with a blind eye to a full end-to-end solution that encompasses all technologies. It’s a dangerous leap of logic exemplified by the article’s subtitle: “Panel at Accelerating Wall Street conference says improving older or poorly written code has greater potential to lower latency than new hardware.”
Don’t get me wrong – optimizing your applications is a critical piece of the puzzle, and speeding up your app might be the single biggest performance gain you can realize. I also agree that throwing new faster Intel boxes at the problem and running the same poor codebase on faster hardware is a poor choice. But as I said in the title, latency is everywhere. If you don’t continuously explore the game-changing potential of new technologies (such as 10GigE, cut-through switches, TCP offload engines and hardware-based messaging, in-memory databases, caching technology) you will forever lag behind competitors who take a holistic approach to latency and invest the time and energy to incorporate important new technologies into their system.
In today’s day and age, you have to explore every piece of technology to achieve the lowest possible latency. There’s just too many exciting developments happening in the areas of networking, middleware and new disk technologies, etc.