The amount of information associated with the following results varies widely. In some cases I personally made the test runs and so am confident of their validity. In many cases I relied on others to make the runs; their responses varied from an email message ("Passed all tests.") to a copy of the program output, but often with limited identification of the hardware system. These results should be regarded as tentative, until full, formal testing can be done.
The machines tested have fallen into one of three more or less clearly defined groups. Some machines were sequentially consistent (SC). This corresponds to an architecture of A(CMP,UPO,PO,CC1). Others obeyed A(CMP,UPO,RR,WW,RW,CC1), that is, they relaxed sequential consistency only in allowing reads to occur before logically preceding writes. Call this the POK architecture, since the IBM mainframes designed in Poughkeepsie were the first to follow this standard. The third group of machines are called processor consistent (PCon) [good89]. They relax both the WR and the CC1 rules. The architecture is A(CMP,UPO,RR,WW,RW,CC3). (More details on the testing can be found in the Appendix.) Given the degree of theoretical interest in relaxed architectures, it is surprising to see how few are the relaxations found on actual machines.
The SC machines were:
A recent study [adgh96] suggests that even more relaxed behavior will be visible on the Alpha, the PowerPC, the T3D, and some other machines.
The results of testing the above 31 machines are given in more detail below. Some number of testers who provided data are not yet willing to be identified.
N=n n is the number of processors on the system.
T=t t is the number of threads in the test run.
K=k k is the length of the arrays used in gathering data.
"." => Test was not run either because too few threads were defined or the test had not been implemented at the time of the test. Tests T1, T2, T4, T7, and T8 run with two threads; the others require more.
"O" => Test was run; strong behavior was observed.
"X" => Test was run; relaxed behavior was observed.
N=4 T=4 K=20000 1991 M. Ayala 1 O. 2 O.. 3 O.. 4 O.. 5 O.. 6 O.. 7 O.. 8 .. 9 .. 11 ... 12 ...
N=3 T=3 K=? 1993 D. Koenen 1 .. 2 O.. 3 ... 4 O.. 5 O.. 6 ... 7 O.. 8 .. 9 .. 11 ... 12 ...
N=4 T=4 K=20000 1994 F. Mounes-Toussi, D. Lilja, Univ. of Minnesota 1 .. 2 OO. 3 OO. 4 OO. 5 OO. 6 OO. 7 OO. 8 O. 9 .. 11 ... 12 ...
N=4 T=4 K=200000 1997 W. Collier 1 OO 2 OOO 3 OOO 4 OOO 5 OOO 6 OOO 7 OOO 8 OO 9 OO 11 OOO 12 OOOThree runs were made with operands forced into read-only state in the cache always (591K), sometimes (582K), or never (595K). All runs showed the same logical behavior. Small variations in the durations of the tests can be seen, but no general pattern is visible.
N=8 T=8 K=500000 1996 N. Manjikian, University of Toronto 1 OO 2 OOO 3 OOO 4 OOO 5 OOO 6 OOO 7 OOO 8 OO 9 OO 11 OOO 12 OOO
N=3 T=3 K=200000 1997 N. Manjikian, University of Toronto 1 OO 2 OOO 3 ... 4 OOO 5 OOO 6 ... 7 OOO 8 OO 9 .. 11 ... 12 ...
N=4 T=4 K=200000 1992 W. Collier 1 O. 2 OO. 3 OO. 4 XX. 5 OO. 6 OO. 7 OO. 8 .. 9 .. 11 ... 12 ...
N=4 T=4 K=200000 1992 W. Collier 1 O. 2 OO. 3 OO. 4 XX. 5 OO. 6 OO. 7 OO. 8 .. 9 .. 11 ... 12 ...
N=2 T=2 K=20000 1995 Anon 1 .. 2 OO. 3 ... 4 XO. 5 ... 6 ... 7 OO. 8 O. 9 .. 11 ... 12 ...
N=2 T=2 K=500000 1995 Anon 1 .. 2 OO. 3 ... 4 XX. 5 ... 6 ... 7 OO. 8 O. 9 .. 11 ... 12 ...
N=2 T=2 K=20000 1995 Anon 1 .. 2 OO. 3 ... 4 XO. 5 ... 6 ... 7 OO. 8 O. 9 .. 11 ... 12 ...
N=2 T=2 K=20000 1995 Anon 1 .. 2 OO. 3 ... 4 OO. 5 ... 6 ... 7 OO. 8 O. 9 .. 11 ... 12 ... N=2 T=2 K=250000 1995 Anon 1 .. 2 OO. 3 ... 4 XX. 5 ... 6 ... 7 OO. 8 O. 9 .. 11 ... 12 ...Two runs were made: a short run (13K) and a long run. (15K). The short run (K = 20000) showed sequentially consistent behavior. The long run (K = 250000) showed behavior which was only rarely and just barely over the edge into relaxed territory.
N=2 T=2 K=200000 1995 Anon 1 OO 2 ... 3 ... 4 XX. 5 ... 6 ... 7 OO. 8 O. 9 .. 11 ... 12 ...
N=2 T=2 K=200000 1998 Ted and Bill Weidenbacher 1 OO 2 OOO 3 ... 4 OXO 5 ... 6 ... 7 OOO 8 OO 9 .. 11 ... 12 ...Three runs were made with operands forced into read-only state in the cache always (225K), sometimes (223K), or never (223K). All runs showed the same logical behavior. Small variations in the durations of the tests can be seen, but no general pattern is visible.
This machine was very close to being sequentially consistent. Only the "never" option showed relaxed behavior, and then only with Test T410, and then only with a distance d of -1. (See Job Run Time Note below*)
N=4 T=4 K=200000 1992 W. Collier 1 O. 2 OO. 3 OO. 4 XX. 5 OO. 6 OO. 7 XX. 8 .. 9 .. 11 ... 12 ...The architecture for this machine is POK; it should not have failed Test T7.
N=2 T=2 K=500000 1995 F. Mounes-Toussi, D. Lilja, Univ. of Minnesota 1 .. 2 OO. 3 ... 4 OX. 5 ... 6 ... 7 OX. 8 O. 9 .. 11 ... 12 ...This machine is SC except when operating on both integer and floating operands. Two runs were made: run1 (19K) and run2 (50K).
N=4 T=4 K=400000 1995 Anon 1 .. 2 OO. 3 OO. 4 XX. 5 OO. 6 OO. 7 XX. 8 O. 9 .. 11 ... 12 ...
N=3 T=3 K=500000 1995 Anon 1 .. 2 OO. 3 OO. 4 XX. 5 OO. 6 OO. 7 XX. 8 O. 9 .. 11 ... 12 ...
N=R T=R K=20000 1995 Anon 1 .. 2 OO. 3 OO. 4 XX. 5 OO. 6 OO. 7 XX. 8 O. 9 .. 11 ... 12 ...
N=2 T=2 K=500000 1995 Anon 1 .. 2 OO. 3 ... 4 XX. 5 ... 6 ... 7 XX. 8 O. 9 .. 11 ... 12 ...
N=3 T=3 K=500000 1995 Anon 1 .. 2 OO. 3 ... 4 XX. 5 OO. 6 ... 7 XX. 8 O. 9 .. 11 ... 12 ...
N=6 T=6 K=250000 1995 Anon 1 .. 2 OO. 3 OO. 4 XX. 5 OO. 6 OO. 7 XX. 8 O. 9 .. 11 ... 12 ...
N=8 T=8 K=20000 1995 M. Olson 1 .. 2 OOO 3 ... 4 XXX 5 ... 6 ... 7 XXX 8 O. 9 .. 11 ... 12 ...
N=2 T=2 K=200000 1997 W. Collier 1 OO 2 OOO 3 ... 4 XXX 5 ... 6 ... 7 XXX 8 OO 9 .. 11 ... 12 ...Three runs were made with operands forced into read-only state in the cache always (313K),sometimes (270K), or never (286K). All runs showed the same logical behavior. Small variations in the durations of the tests can be seen, but the variations are different for different tests. No general pattern is apparent.
N=2 T=2 K=200000 1998 W. Collier 1 OO 2 OOO 3 ... 4 XXX 5 ... 6 ... 7 XXX 8 OO 9 .. 11 ... 12 ...Three runs were made with operands forced into read-only state in the cache always (237K),sometimes (231K), or never (287K). All runs showed the same logical behavior. Small variations in the durations of the tests can be seen, but the variations are different for different tests. No general pattern is apparent.
N=2 T=2 K=500000 1998 Jim Reilly, Aqua Process Corp. 1 OO 2 OOO 3 ... 4 XXX 5 ... 6 ... 7 XXX 8 OO 9 .. 11 ... 12 ...Three runs were made with operands forced into read-only state in the cache always (259K),sometimes (252K), or never (256K). All three runs showed the same logical behavior. This is the most aggressively relaxed machine seen so far. Values of d = -25 were seen for tests T4 and T7. (See Job Run Time Note below*)
N=2 T=4 K=500000 2004 Brad Richards 1 OO 2 OOO 3 OOO 4 XXX 5 OOO 6 OOO 7 XXX 8 OO 9 OO 11 XXX 12 XXX
N=8 T=4 K=500000 2004 Brad Richards 1 OO 2 OOO 3 OOO 4 XXX 5 OOO 6 OOO 7 XXX 8 OO 9 OO 11 XXX 12 XXX
N=2 T=2 K=500000 1998 Dave Magram, at the HP Booth at PC Expo. 1 OO 2 OOO 3 ... 4 OOO 5 ... 6 ... 7 OOO 8 OO 9 .. 11 ... 12 ...What initially appeared to be SC results turned out on closer inspection to be more prosaic. It appears that only one processor was active. Thus there was no simultaneous execution and so no chance to see relaxed behavior. The machine was fast. The test output is displayed to show just how fast. (See Job Run Time Note below*)
*Job Run Time Note. The minimum run time for a job in the end of run report is shown as 0.5 seconds. This is not accurate. Accurate run time information can be found in the data for each test. The source of this misinformation lies in a degradation of function in Visual C++ Version 5 as compared to an earlier version. More accurate information will be available in the future.
Last updated January 4, 2006.