C-Synthesis Benchmarks revisited -- and a proposed design for the benchmark suite

A few posts ago, I talked about a new benchmark suite being proposed for C-synthesis. I've had a chance to look at one of the benchmarks, specifically the MIPS design. Based on this benchmark, I think it's worth taking a deeper look at some of the other benchmarks.

The MIPS design is basically a small piece of C code that executes a subset of MIPS processor-like instructions -- it's basically an instruction set simulator built mostly as a large case statement on the ops code of the instruction. I'm curious what the synthesis results for this design would mean -- there is no pipeline, no microprocessor architecture, nor would one expect that a C synthesis tool would generate one from this code. Additionally, it's not a particularly complicated design. As a control based example, I'm not sure it would tell you much.

If we are going to have a benchmark suite, it needs to reflect the complexity and functionality of real designs that people would do -- and ideally be of a size that allows design teams to understand differences. A good example of a complex design suitable to benchmarking is Reed-Solomon, which has:

• Complexity
• Decent size
• Practical use -- it's a design that someone might really need to implement
  

On OpenCores, there are publicly available sources available to Reed-Solomon for both C and Bluespec. And, as the C version was designed with synthesis in mind, it would be the perfect candidate for assessing C synthesis tools (and can be used to compare against what's achievable with a Bluespec design). The entire project and code sets are located at OpenCores (with the C reference code available in the sw-reedsolomon directory of the SVN repository for the project). You must be logged in to OpenCores to see SVN. The project is called bluespec-reedsolomon.�

Hopefully, CHStone might consider Reed-Solomon in its mix. If I were looking at C synthesis, I'd take a look at this design.