Fix flaky tests in DataFieldConverterTest.java

[FanYuliang]

Hi,

Since there was no response to #866, I'm also opening this PR but can revise it if you want.
Also, this PR is related to #870, but fixed more flaky tests in DataFieldConverterTest.java

By running the following command:
mvn install -DskipTests,
mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex -Dtest=de.neemann.digital.core.memory.DataFieldConverterTest

Three tests testMarshalObj, testMarshalObj2, testMarshalObj3 in de.neemann.digital.core.memory.DataFieldConverterTest will fail sometimes.

The reason is that method xStream.toXML() in com.thoughtworks.xstream.XStream is not deterministic: Xstreamer.toXML(object) method first serializes the object using HashMap and then convert the XML object to string in the order of keys in the HashMap. I fixed the flakiness by checking whether the actual output is in the expected list, which contains all the possible XML combination orders.

[hneemann]

I have not yet thought about how to deal with this problem. You call these tests "flaky" but actually they are not, they have been running stable for years without any problems.
I haven't gotten around to looking at why that is.

[FanYuliang]

Hi hneemann,

Thank you for your response! It's interesting that when I run Nondex, which is a flaky test detector for java projects by rerunning the tests, it says testMarshalObj, testMarshalObj2, testMarshalObj3 will sometimes fail during the runs. However, the first run of test never fails. I will dive deep and cehck if it's the Nondex issue or the source code. I will let you if I have any findings.

[hneemann]

I have tested this using a class with a huge number of field in random order of declaration.
If such a class is serialized the fields appear in the exact same order in the xml file as they are declared in the class.
There is no randomization going on during serialization.

Therefore, I think the analysis tool is buggy.

    private static final class Flaky {
        int a = 1;
        int x = 2;
        int f = 3;
        int e = 4;
        int v = 5;
        int l = 6;
        int k = 7;
        int s = 8;
        int c = 9;
        int j = 10;
        int n = 11;
        int g = 12;
        int w = 13;
        int r = 14;
        int m = 15;
        int b = 16;
        int p = 17;
        int o = 18;
        int y = 19;
        int t = 20;
        int h = 21;
        int u = 22;
        int d = 23;
        int z = 24;
        int i = 25;
        int q = 26;
    }

    public void testFlaky() {
        XStream xStream = getxStream();
        xStream.alias("flaky", Flaky.class);
        assertEquals("<?xml version=\"1.0\" ?>" +
                        "<flaky>" +
                        "<a>1</a>" +
                        "<x>2</x>" +
                        "<f>3</f>" +
                        "<e>4</e>" +
                        "<v>5</v>" +
                        "<l>6</l>" +
                        "<k>7</k>" +
                        "<s>8</s>" +
                        "<c>9</c>" +
                        "<j>10</j>" +
                        "<n>11</n>" +
                        "<g>12</g>" +
                        "<w>13</w>" +
                        "<r>14</r>" +
                        "<m>15</m>" +
                        "<b>16</b>" +
                        "<p>17</p>" +
                        "<o>18</o>" +
                        "<y>19</y>" +
                        "<t>20</t>" +
                        "<h>21</h>" +
                        "<u>22</u>" +
                        "<d>23</d>" +
                        "<z>24</z>" +
                        "<i>25</i>" +
                        "<q>26</q>" +
                        "</flaky>",
                xStream.toXML(new Flaky()));
    }

[FanYuliang]

Hi hneemann,

Looks like the experiment shows there's nothing wrong in the XStream. I'll look into the flaky test detector and see if there's any bug in the tool. I'll close this PR for now and will let you know once I figure it out.

[FanYuliang]

Hi, sorry for the late reply,

NonDex reruns the test by shuffling the order of the HashMap to simulate the indeterministic feature of HashMap. Even though based on the my JVM and the feedback from you which indicates that there's no flakiness in these tests, it doesn't mean the tests are not "flaky" by the definition. Because xStream.toXML() uses HashMap to marshal objects, flakiness might come in the future/ other JVMs. That's why I suggest to fix the flaky test.