One of the challenges with multichannel data acquisition (DAQ) systems is the aggregation of all the channels to a single processing unit. As the number of channels or the acquisition bandwidth increases, the challenge gets bigger and bigger. In this blog post, I look at the advantages and drawbacks of two system topologies when it comes to channel aggregation: a single-width MicroTCA system with a switched topology and a double-width MicroTCA system with a point-to-point topology.
In this post, a 128-channel system will be used as an example. The MI125-32, a 32-channel FPGA mezzanine card (FMC) stack, along with a Perseus FPGA carrier card (containing a Virtex-6 FPGA), will be used in all the systems. The MI125 analog-digital convertor (ADC) operates at 125 Msps and offers a sample resolution of 14 bits. Its data generation capability is therefore 1.75 Gigabits per second per channel, or 56 Gbps for 32 channels.
Single-width MicroTCA system with switched topology
The Perseus601x is used to demonstrate the switched topology system. The Perseus601x has one high-pin count FMC site and therefore can hold one MI125-32 FMC stack for a total of 32 channels. The system consists of four Perseus601x cards in a MicroTCA chassis. In this configuration, the data pipe for the channels is the PCI Express bus on the backplane.
The first drawback of this topology is the number of sampling elements. All the channels from the three Perseus601x cards must be transferred to the fourth in order to aggregate all the channels on the same FPGA. The necessary incoming throughput of the fourth FPGA would need to be 3 x 56 Gbps (the 32 channel data mass calculated above) for 168 Gbps.
The Perseus601x-to-Perseus601x PCIe peak data rate has been measured at 815 Mbytes per second or 6.52 Gbps. Using a single channel throughput of 1.75 Gbps, only three channels can be sent from any of the first three Perseus cards to the fourth. In this case, the switched topology creates the bottleneck; all data has to go through the switch. The only way to increase the number of transferable channels is to pre-process the acquired signal.
Double-width MicroTCA system with point-to-point topology
A double-width MicroTCA blade like the Perseus611x can hold two MI125-32 FMCs. The Perseus611x has two high-pin count FMC sites and therefore can hold two MI125-32 FMC stacks for a total of 64 channels. The system consists of two Perseus611x cards in a MicroTCA.4 chassis.
The difference with this system, as compared to the switched topology, is that the double-width form factor and the MicroTCA.4 chassis enables the use of rear-transition modules (RTM). In this case, each RTM holds seven mini-SAS connectors which expose four Gigabit transceivers (GTX) each, thus creating a very high data rate path. If we assume that 28 GTX are used at an efficient 3.5 Gbps, the aggregate throughput reaches 98 Gbps in each direction.
Using the same 1.75 Gbps channel throughput, up to 56 channels can therefore be sent at full bandwidth from one FPGA to the other. In this case, each FPGA would be able to process 120 of the 128 channels.
As the Perseus611x uses only one Virtex-6 FPGA to sample all 64 channels, the drawback of this proposed architecture is that the processing power of the system is divided by two (four FPGAs for the switched topology versus two for the point-to-point topology).