(−1)–ENUMERATION OF SELF–COMPLEMENTARY PLANE PARTITIONS

THERESIA EISENKÖLBL

Institut für Mathematik der Universität Wien, Nordbergstraße 15, A-1090 Wien, Austria.

                      E-mail: Theresia.Eisenkoelbl@univie.ac.at

Abstract. We prove a product formula for the remaining cases of the weighted enumeration of self–complementary plane partitions contained in a given box where adding one half of an orbit of cubes and removing the other half of the orbit changes the sign of the weight. We use nonintersecting lattice path families to express this enumeration as a Pfaffian which can be expressed in terms of the known ordinary enumeration of self–complementary plane partitions.

1. Introduction

A plane partition P can be defined as a finite set of points (i, j, k) with i, j, k > 0 and if (i, j, k) ∈ P and 1 ≤ i′ ≤ i, 1 ≤ j′ ≤ j, 1 ≤ k′ ≤ k then (i′, j′, k′) ∈ P. We interpret these points as midpoints of cubes and represent a plane partition by stacks of cubes (see Figure 1). If we have i ≤ a, j ≤ b and k ≤ c for all cubes of the plane partition, we say that the plane partition is contained in a box with sidelengths a, b, c.

Plane partitions were first introduced by MacMahon. One of his main results is the following [10, Art. 429, x → 1, proof in Art. 494]:

The number of all plane partitions contained in a box with sidelengths a, b, c equals

B(a, b, c) = ∏_{i=1}^{a} ∏_{j=1}^{b} ∏_{k=1}^{c} \frac{i + j + k − 1}{i + j + k − 2} = ∏_{i=1}^{a} \frac{(c + i)_{b}}{(i)_{b}} ,  (1)

where (a)_{n} := a(a + 1)(a + 2) … (a + n − 1) is the rising factorial.

MacMahon also started the investigation of the number of plane partitions with certain symmetries in a given box. These numbers can also be expressed as product formulas similar to the one given above. In [14], Stanley introduced additional complementation symmetries giving six new combinations of symmetries which led to more conjectures all of which were settled in the 1980’s and 90’s (see [14, 8, 3, 17]).

Many of these theorems come with q–analogs, that is, weighted versions that record the number of cubes or orbits of cubes by a power of q and give expressions containing q–rising factorials instead of rising factorials (see [1, 2, 11]). For plane partitions with complementation symmetry, it seems to be difficult to find natural q–analogs. However, in Stanley’s paper a q–analog for self–complementary plane partitions is given (the

2000 Mathematics Subject Classification. Primary 05A15; Secondary 05B45 52C20.

Key words and phrases. lozenge tilings, rhombus tilings, plane partitions, determinants, pfaffians, nonintersecting lattice paths.

Figure 1. A self–complementary plane partition

weight is not symmetric in the three sidelengths, but the result is). Interestingly, upon setting q = −1 in the various q–analogs, one consistently obtains enumerations of other objects, usually with additional symmetry restraints. This observation, dubbed the “(-1) phenomenon” has been explained for many but not all cases by Stembridge (see [15] and [16]).

In [7], Kuperberg defines a (−1)–enumeration for all plane partitions with complementation symmetry which admits a nice closed product formula in almost all cases. These conjectures were solved in Kuperberg’s own paper and in the paper [4] except for one case without a nice product formula and the case of self-complementary plane partitions in a box with some odd sidelengths which will be the main theorem of this paper. We start with the precise definitions for this case.

A plane partition P contained in the box a × b × c is called self–complementary if (i, j, k) ∈ P ⇔ (a + 1 − i, b + 1 − j, c + 1 − k) ∉ P for 1 ≤ i ≤ a, 1 ≤ j ≤ b, 1 ≤ k ≤ c. This means that one can fill up the entire box by placing the plane partition and its mirror image on top of each other. A convenient way to look at a self–complementary plane partition is the projection to the plane along the (1, 1, 1)–direction (see Figure 1). A plane partition contained in an a × b × c–box becomes a rhombus tiling of a hexagon with sidelengths a, b, c, a, b, c. It is easy to see that self-complementary plane partitions correspond exactly to those rhombus tilings with a 180^{∘} rotational symmetry.

The (−1)–weight is defined as follows: A self–complementary plane partition contains exactly one half of each orbit under the operation (i, j, k) ↦ (a+1−i, b+1−j, c+1−k). Let a move consist of removing one half of an orbit and adding the other half. Two plane partitions are connected either by an odd or by an even number of moves, so it is possible to define a relative sign. The sign becomes absolute if we assign weight 1 to the half-full plane partition (see Figure 2 for a box with one side of even length and Figure 5 for a box with two).

Therefore, this weight is (−1)^{n(P)} where n(P) is the number of cubes in the “left” half of the box (after cutting through the sides of length a) if a is even and b, c odd

Figure 2. A plane partition of weight 1.

or the number of cubes in the upper half of the box (after cutting through the sides of length b) if a is odd and b, c are even and we want to evaluate ∑_{P} (−1)^{n(P)}. For example, the plane partition in Figure 1 has weight (−1)^{10} = 1.

In order to be able to state the result for the (−1)–enumeration more concisely, Stanley’s result on the ordinary enumeration of self–complementary plane partitions is needed. It will also be used as a step in the proof of the (−1)–enumeration.

Theorem 1 (Stanley [14]). The number SC(a, b, c) of self–complementary plane partitions contained in a box with sidelengths a, b, c can be expressed in terms of B(a, b, c) in the following way:

B (\frac{a}{2}, \frac{b}{2}, \frac{c}{2})^{2} for a, b, c even,
B (\frac{a}{2}, \frac{b+1}{2}, \frac{c−1}{2}) B (\frac{a}{2}, \frac{b−1}{2}, \frac{c+1}{2}) for a even and b, c odd,
B (\frac{a+1}{2}, \frac{b}{2}, \frac{c}{2}) B (\frac{a−1}{2}, \frac{b}{2}, \frac{c}{2}) for a odd and b, c even,

where B(a, b, c) = ∏_{i=1}^{a} \frac{(c+i)_{b}}{(i)_{b}} is the number of all plane partitions in an a × b × c–box.

Note that a self-complementary plane partitions contains exactly half of all cubes in the box. Therefore, there are no self-complementary plane partitions in a box with three odd sidelengths.

Now we can express the (−1)–enumeration of self–complementary plane partitions in terms of SC(a, b, c), the ordinary enumeration of self–complementary plane partitions.

Theorem 2. The enumeration of self–complementary plane partitions in a box with sidelengths a, b, c counted with weight (−1)^{n(P)} equals up to sign

B (\frac{a}{2}, \frac{b}{2}, \frac{c}{2}) for a, b, c even,
SC (\frac{a}{2}, \frac{b+1}{2}, \frac{c−1}{2}) SC (\frac{a}{2}, \frac{b−1}{2}, \frac{c+1}{2}) for a even and b, c odd
SC (\frac{a+1}{2}, \frac{b}{2}, \frac{c}{2}) SC (\frac{a−1}{2}, \frac{b}{2}, \frac{c}{2}) for a odd and b, c even

where SC(a, b, c) is given in Theorem 1 in terms of the numbers of plane partitions contained in a box and n(P) is the number of cubes in the plane partition P that are not in the half-full plane partition (see Figure 2 and 5).

Remark. Note that this is zero for exactly the cases a ≡ 2 (mod 4), b ≢ c (mod 4) or a odd, b ≡ c ≡ 2 (mod 4) (because then the three parameters of one factor are odd). This includes the cases where it changes the weight if we assign 1 to another ”half-full” plane partition.

Since the sides of the box play symmetric roles this covers all cases. (For three odd sidelengths there are no self-complementary plane partitions.) The case of three even sidelengths has already been proved in [4].

In Stanley’s paper [14], the theorem actually gives a q–enumeration of plane partitions. The case q = −1 gives the same result as the theorem above if one or more side has odd length, but for even sidelengths, Stanley’s theorem gives SC (a/2, b/2, c/2)^{2} which does not equal B (a/2, b/2, c/2). While the result is the same if some of the sidelengths are odd, the weights of individual plane partitions are different. Outline of the proof Step 1: From plane partitions to families of nonintersecting lattice paths.

First, we adjust a well-known bijection between plane partitions and families of nonintersecting lattice paths to rephrase the problem as a path enumeration problem (see Figure 3 to get an idea). Step 2: From lattice paths to a sum of Pfaffians

By the main theorem on nonintersecting lattice paths (see Lemma 3), this enumeration can be expressed as a sum of determinants (see Lemma 4). Step 3: The sum of determinants is a single Pfaffian

This sum can be expressed as a Pfaffian (see Lemma 7) by a theorem of Ishikawa and Wakayama (see Lemma 5). An analogous expression can be written down for the ordinary enumeration of self-complementary plane partitions (see Lemma 8). Step 4: Evaluation of the Pfaffian

Finally, the matrix is transformed to a block matrix by elementary row and column operations. Here, it becomes necessary to do a case-by-case analysis according to the parity of the parameters, but the general idea is the same in all cases. The original entries contain expressions with (−1)–binomial coefficients which are either zero or ordinary binomial coefficients with parameters of half the size (see (5)). The row and column operations involve separating (combinations of) the even- and odd-numbered rows and columns. Therefore, the two blocks we obtain have the same structure as the original matrix, but the (−1)–binomial coefficients are replaced by ordinary binomial coefficients.

} { b { {  } c − x a x  a + b

• _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ∘ ∘ ∘ ∘ ∘ ∘ ∘ ∘^{A_{1}} ^{A_{2}} ^{A_{3}} ^{A_{4}}^{E_{2}} ^{E_{3}}  ^{E_{5}} ^{E_{6}}

Figure 3. The paths for the self–complementary plane partition in Figure 1 and the orthogonal version. (x = \frac{c−b}{2})

Now, we can identify this as certain instances of the ordinary enumeration of selfcomplementary plane partitions. Since closed-form expressions for these are already given by Stanley (see Theorem 1), we can immediately derive the theorem.

2. Proof

Step 1: From plane partitions to families of nonintersecting lattice paths.

We use the projection to the plane along the (1, 1, 1)–direction and get immediately that self–complementary plane partitions contained in an a × b × c–box are equivalent to rhombus tilings of a hexagon with sides a, b, c, a, b, c invariant under 180^{∘}–rotation. A tiling of this kind is clearly determined by one half of the hexagon.

Since the sidelengths a, b, c play a completely symmetric role and two of them must have the same parity we assume without loss of generality that c − b is even and b ≤ c. The result turns out to be symmetric in b and c, so we can drop the last condition in the statement of Theorem 2. Write x for the positive integer (c − b)/2 and divide the hexagon in half with a line parallel to the side of length a (see Figure 3). As shown in the same figure, we find a bijection between these tiled halves and families of nonintersecting lattice paths.

The starting points of the lattice paths are the midpoints of the edges on the side of length a. The end points are the midpoints of the edges parallel to a on the opposite boundary. This is a symmetric subset of the midpoints on the cutting line of length a + b.

The paths always follow the rhombi of the given tiling by connecting midpoints of parallel rhombus edges. It is easily seen that the resulting paths have no common points (i.e. they are nonintersecting) and the tiling can be recovered from a nonintersecting lattice path family with unit diagonal and down steps and appropriate starting and end points. Of course, the path families will have to be counted with the appropriate (−1)–weight.

} { b { {  } c − x a x  a + b

• _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ∘ ∘ ∘ ∘ ∘ ∘ ∘ ∘ ∘ ∘^{A_{1}} ^{A_{2}} ^{A_{3}} ^{A_{4}} ^{A_{5}}_{E_{1}}  ^{E_{3}} ^{E_{4}} ^{E_{5}}  ^{E_{7}}

Figure 4. The paths for a self–complementary plane partitions with odd a. (x = \frac{c−b}{2})

Figure 5. A plane partition of weight 1 with odd a.

After changing to an orthogonal coordinate system (see Figure 3), the paths are composed of unit South and East steps and the coordinates of the starting points are

A_{i} = (i − 1, b + i − 1) for i = 1, …, a.  (2)

The end points are a points chosen symmetrically among

E_{j} = (x + j − 1, j − 1) for j = 1, …, a + b.  (3)

Here, symmetrically means that if E_{j} is chosen, then E_{a+b+1−j} must be chosen as well.

Note that the number a + b of potential end points on the cutting line is always odd. Therefore, there is a middle one which is either in all path families or in none according to the parity of a (see Figures 3 and 4).

Now the (−1)–weight has to be defined for the paths. For a path from A_{i} to E_{j} we can use the weight (−1)^{area(P)} where area(P) is the area between the path and the x–axis and then multiply the weights of all the paths. We have to check that the weight changes sign if we replace a half orbit with the complementary half orbit. If one of the affected cubes is completely inside the half shown in Figure 3 or 4, ∑_{P} area(P) changes by one. If the two affected cubes are on the border of the figure, two symmetric endpoints, say E_{j} and E_{a+b+1−j}, are changed to E_{j+1} and E_{a+b−j} or vice versa. It is easily checked that in this case ∑_{P} area(P) changes by j + (a + b − j) which is odd.

It is straightforward to check that the weight for the “half-full” plane partition (see Figures 2 and 5) equals (−1)^{a(a−2)/8} for a even, b, c odd, and (−1)^{(a+b−1)c/4} for a odd, b, c even. Therefore, we have to multiply the path enumeration by the respective global sign. Step 2: From lattice paths to a sum of Pfaffians

This weight can be expressed as a product of weights on individual steps (the exponent of (−1) is just the height of the step), so the following lemma is applicable. By the main theorem on nonintersecting lattice paths (see [9, Lemma 1] or [5, Theorem 1]) the weighted count of such families of paths can be expressed as a determinant.

Lemma 3. Let A_{1}, A_{2}, …, A_{n}, E_{1}, E_{2}, …, E_{n} be integer points meeting the following condition: Any path from A_{i} to E_{l} has a common vertex with any path from A_{j} to E_{k} for any i, j, k, l with i < j and k < l.

Then we have

P(A → E, nonint.) = det_{1≤i,j≤n} (P(A_{i} → E_{j})),  (4)

where P(A_{i} → E_{j}) denotes the weighted enumeration of all paths running from A_{i} to E_{j} and P(A → E, nonint.) denotes the weighted enumeration of all families of nonintersecting lattice paths running from A_{i} to E_{i} for i = 1, …, n.

The condition on the starting and end points is fulfilled in our case because the points lie on diagonals, so we have to find an expression for T_{ij} = P(A_{i} → E_{j}), the weighted enumeration of all single paths from A_{i} to E_{j} in our problem.

It is well-known that the enumeration of paths of this kind from (x, y) to (x′, y′) is given by the q–binomial coefficient \left[{x′−x+y−y′ \atop x′−x}\right]_{q} if the weight of a path is q^{e} where e is the area between the path and a horizontal line through its endpoint.

The q–binomial coefficient (see [13, p. 26] for further information) can be defined as

\left[{n \atop k}\right]_{q} = \frac{∏_{j=n−k+1}^{n}(1 − q^{j})}{∏_{j=1}^{k}(1 − q^{j})}.

Although it is not obvious from this definition, the q–binomial coefficient is a polynomial in q. So it makes sense to put q = −1.

It is easy to verify that

\left[{n \atop k}\right]_{−1} = \begin{cases}0 & n even, k odd, \\ \binom{⌊n/2⌋}{⌊k/2⌋} & else. \end{cases}  (5)

Taking also into account the area between horizontal line through the endpoint and the x–axis, we obtain

T_{ij} = P(A_{i} → E_{j}) = (−1)^{(x+j−i)(j−1)}\left[{b + x \atop b + i − j}\right]_{−1}.

Now we apply Lemma 3 to all possible sets of end points. Thus, the (−1)–enumeration can be expressed as a sum of determinants which are minors of the a × (a + b)–matrix T:

Lemma 4. The (−1)–enumeration can be written as

(−1)^{a(a−2)/8} ∑_{1≤k_{1}<···<k_{a/2}≤(a+b−1)/2} det(T_{k_{1}}, …, T_{k_{a/2}}, T_{a+b+1−k_{a/2}}, …, T_{a+b+1−k_{1}})

for a even and b, c odd,

(−1)^{c(a+b−1)/4} ∑_{1≤k_{1}<···<k_{(a−1)/2}≤(a+b−1)/2} det(T_{k_{1}}, …, T_{k_{(a−1)/2}}, T_{(a+b+1)/2}, T_{a+b+1−k_{(a−1)/2}}, …, T_{a+b+1−k_{1}})

for a odd and b, c even,

where T_{ij} is (−1)^{(x+j−i)(j−1)}\left[{b+x \atop b+i−j}\right]_{−1} and T_{j} denotes the jth column of T which has length a.

Remark. The same argument works for the ordinary enumeration, we just have to replace T_{ij} by the ordinary enumeration \binom{b+x}{b+i−j}.

Step 3: The sum of determinants is a single Pfaffian Recall that the Pfaffian of a skew–symmetric 2n × 2n–matrix M is defined as

Pf M = ∑_{m} sgn m ∏_{\begin{matrix}\{i,j\}∈m \\ i<j \end{matrix}} M_{ij},

where the sum runs over all m = {{m_{1}, m_{2}}, {m_{3}, m_{4}}, …, {m_{2n−1}, m_{2n}}} with the conditions {m_{1}, …, m_{2n}} = {1, …, 2n}, m_{2k−1} < m_{2k} and m_{1} < m_{3} < ⋯ < m_{2n−1}. The term sgn m is the sign of the permutation m_{1}m_{2}m_{3} … m_{2n}.

We will use the fact that (Pf M)^{2} = det M and that simultaneous row and column operations have the same effect on the Pfaffian as ordinary row or column operations on the determinant.

Our sums of determinants can be simplified by a theorem of Ishikawa and Wakayama [6, Theorem 1(1)] which we use to express the sum as a Pfaffian. Our way of stating the theorem is taken from [12, Corollary 3.2].

Lemma 5. Suppose that n ≤ p and n is even. Let T = (t_{ik}) be a p × n matrix and A = (a_{kl}) be a p × p skew-symmetric matrix. Then we have

∑_{1≤k_{1}<···<k_{n}≤p} Pf (A_{k_{1},...,k_{n}}^{k_{1},...,k_{n}}) det(T_{k_{1},...,k_{n}}) = Pf(^{t}T AT),

where ^{t}T denotes the transpose of the matrix T, T_{k_{1},...,k_{n}} is the matrix composed of the rows of T with indices k_{1}, …, k_{n} and A_{k_{1},...,k_{n}}^{k_{1},...,k_{n}} is the matrix composed of the rows and columns of A with indices k_{1}, …, k_{n}.

(−1)–ENUMERATION OF SELF–COMPLEMENTARY PLANE PARTITIONS 9

Now specialize to A = (\begin{matrix}0 & I_{n} \\ −I_{n} & 0 \end{matrix}). Lemma 6. Let S be a 2m × 2n–matrix with m ≤ n and S^{∗} be the matrix

(S_{1}, …, S_{n}, S_{2n}, …, S_{n+1})

where S_{j} denotes the jth column of S. Let A be the matrix (\begin{matrix}0 & I_{n} \\ −I_{n} & 0 \end{matrix}). Then the following identity holds:

∑_{1≤k_{1}<···<k_{m}≤n} det(S_{k_{1}}, …, S_{k_{m}}, S_{2n+1−k_{m}}, …, S_{2n+1−k_{1}}) = Pf(S^{∗}A(^{t}S^{∗}))

= Pf_{1≤i,j≤2m} (∑_{k=1}^{n} (S_{ik}S_{j,2n+1−k} − S_{jk}S_{i,2n+1−k})) .

Proof. The proof follows from Lemma 5 with A = (\begin{matrix}0 & I_{n} \\ −I_{n} & 0 \end{matrix}) and T = ^{t}S^{∗}. The sign of Pf (A_{k_{1},...,k_{m},k_{1}+n,...,k_{m}+n}^{k_{1},...,k_{m},k_{1}+n,...,k_{m}+n}) cancels exactly with the sign obtained from the reordering of the columns of S in the determinant. □

Now we apply this lemma to our sums. Lemma 7. The Pfaffians for the various (−1)–enumerations for b ≤ c are

(−1)^{a(a−2)/8} Pf_{1≤i,j≤a} ( ∑_{k=1}^{\frac{a+b−1}{2}} (T_{ik}T_{j,a+b+1−k} − T_{jk}T_{i,a+b+1−k})) ,

for a even and b, c odd,

(−1)^{c(a+b−1)/4+(a−1)/2} Pf_{1≤i,j≤a+1} ( \begin{matrix}∑_{k=1}^{\frac{a+b−1}{2}}(T_{ik}T_{j,a+b+1−k} − T_{jk}T_{i,a+b+1−k}) & T_{i, \frac{a+b+1}{2}} \\ −T_{j, \frac{a+b+1}{2}} & 0 \end{matrix} )

for a odd and b, c even,

where T_{ij} = (−1)^{(x−i)(j−1)}\left[{b+x \atop b+i−j}\right]_{−1} (and x = (c − b)/2). Proof. In the first case, apply the lemma with 2m = a, 2n = a + b − 1 and S = (T_{1}, …, T_{\frac{a+b−1}{2}} , T_{\frac{a+b+3}{2}} , …, T_{a+b}) to obtain

Pf_{1≤i,j≤a} ( ∑_{k=1}^{\frac{a+b−1}{2}} (T_{ik}T_{j,a+b+1−k} − T_{jk}T_{i,a+b+1−k})) .

In the second case, apply the lemma with 2m = a + 1, 2n = a + b + 1 and

S = (\begin{matrix}T_{1} & … & T_{(a+b+1)/2} & 0 0 ⋮ 0 & T_{(a+b+3)/2} & … & T_{a+b} \\ 0 & … & 0 & 1 & 0 & … & 0 \end{matrix}) ,

where the T_{j} are columns of length a. We get

Pf_{1≤i,j≤a+1} ( \begin{matrix}∑k=1^{\frac{a+b−1}{2}} (T_{ik}T_{j,a+b+1−k} − T_{jk}T_{i,a+b+1−k}) & T_{i, \frac{a+b+1}{2}} \\ −T_{j, \frac{a+b+1}{2}} & 0 \end{matrix} ) .

(The extra row and column correspond to an extra starting point A_{a+1} and extra end point E_{a+b+1} which are only connected to each other, so this end point must be chosen. This forces the choice of E_{(a+b+1)/2} and also gives an additional sign of (−1)^{(a−1)/2}). □

Lemma 8. The Pfaffians for the ordinary enumeration SC(a, b, c) for b ≤ c are

Pf_{1≤i,j≤a} (\begin{matrix}⌊ & \frac{a+b}{2} & ⌋ \\ ∑ \\ k=1 \end{matrix} (\binom{b + x}{b + i − k}\binom{b + x}{j + k − a − 1} − \binom{b + x}{b + j − k}\binom{b + x}{i + k − a − 1}))

for a and c − b even

(−1)^{(a−1)/2} Pf_{1≤i,j≤a+1} ( \begin{matrix}∑_{k=1}^{\frac{a+b−1}{2}}(\binom{b+x}{b+i−k}\binom{b+x}{j+k−a−1} − \binom{b+x}{b+j−k}\binom{b+x}{k+i−a−1}) & \binom{b+x}{b+i− \frac{a+b+1}{2}} \\ −\binom{b+x}{b+j− \frac{a+b+1}{2}} & 0 \end{matrix} )

for a odd and b, c even.

Proof. Replace T_{ij} by the ordinary enumeration of the respective paths. This replaces (−1)–binomial coefficients by ordinary ones. (Doing the same thing for the analogous expressions in Section 9 of [4] gives the result for the case of even sidelengths.) □

Remark. Of course, the closed form of these Pfaffians is known by Stanley’s theorem (see Theorem 1). Therefore, we can use them to evaluate the Pfaffians for the (−1)– enumeration.

Step 4: Evaluation of the Pfaffian

Now, the Pfaffians of Lemma 7 can be reduced to products of the known Pfaffians corresponding to the ordinary enumeration. We have to do the calculations separately for different parities of the parameters.

Case a, x even, b, c odd We are in the first case of Lemma 7. For M_{ij} in Pf M we can write

∑_{k=1}^{(a+b−1)/2}(−1)^{(k+1)(i+j)} (\binom{(b + x − 1)/2}{⌊(b + i − k)/2⌋}\binom{(b + x − 1)/2}{⌊(j + k − a − 1)/2⌋}

−\binom{(b + x − 1)/2}{⌊(b + j − k)/2⌋}\binom{(b + x − 1)/2}{⌊(i + k − a − 1)/2⌋})

with 1 ≤ i, j ≤ a.

Splitting the sum into terms k = 2l and k = 2l − 1 gives

∑_{l=1}^{⌊(a+b−1)/4⌋}(−1)^{i+j} (\binom{(b + x − 1)/2}{(b − 1)/2 + ⌊(i − 1)/2⌋ − l + 1}\binom{(b + x − 1)/2}{⌊(j − 1)/2⌋ + l − a/2}

−\binom{(b + x − 1)/2}{(b − 1)/2 + ⌊(j − 1)/2⌋ − l + 1}\binom{(b + x − 1)/2}{⌊(i − 1)/2⌋ + l − a/2})

+ ∑_{l=1}^{⌈(a+b−1)/4⌉} (\binom{(b + x − 1)/2}{(b − 1)/2 + ⌊i/2⌋ − l + 1}\binom{(b + x − 1)/2}{⌊j/2⌋ + l − a/2 − 1}

−\binom{(b + x − 1)/2}{(b − 1)/2 + ⌊j/2⌋ − l + 1}\binom{(b + x − 1)/2}{⌊i/2⌋ + l − a/2 − 1}) (6)

Now we apply some row and column operations to our matrix M. Start with row(1), then write the differences row(2i + 1) − row(2i) for i = 1, …, a/2 − 1, and finally row(2i−1)+row(2i) for i = 1, …, a/2. Now apply the same operations to the columns, so that the resulting matrix is still skew–symmetric. The new matrix has the same Pfaffian only up to sign (−1)^{(a/2)(a/2−1)/2} which cancels with the global sign in Lemma 7.

Computation gives:

M_{2i+1,j} − M_{2i,j}

= − ∑_{l=1}^{⌊(a+b−1)/4⌋}(−1)^{j} (\binom{(b + x − 1)/2 + 1}{(b − 1)/2 + i − l + 1}\binom{(b + x − 1)/2}{⌊(j − 1)/2⌋ + l − a/2}

−\binom{(b + x − 1)/2}{(b − 1)/2 + ⌊(j − 1)/2⌋ − l + 1}\binom{(b + x − 1)/2 + 1}{i + l − a/2})

Thus, apart from the first row and column, the left upper corner looks like

M_{2i+1,2j+1} − M_{2i,2j+1} − M_{2i+1,2j} + M_{2i,2j}

= ∑_{l=1}^{⌊(a+b−1)/4⌋} (\binom{(b + x − 1)/2 + 1}{(b − 1)/2 + i − l + 1}\binom{(b + x − 1)/2 + 1}{j + l − a/2}

−\binom{(b + x − 1)/2 + 1}{(b − 1)/2 + j − l + 1}\binom{(b + x − 1)/2 + 1}{i + l − a/2}) , (7)

where i, j = 1, … a/2 − 1. Note how similar this is to the original matrix, only the (−1)–binomial coefficients are now replaced with ordinary binomial coefficients. The goal is to identify two blocks in the matrix which correspond to ordinary enumeration of self–complementary plane partitions.

The right upper corner is zero (of size (a/2 − 1) × a/2).

Furthermore,

M_{2i−1,j} + M_{2i,j} = ∑_{l=1}^{⌈(a+b−1)/4⌉} (\binom{(b + x − 1)/2 + 1}{(b − 1)/2 + i − l + 1}\binom{(b + x − 1)/2}{⌊j/2⌋ + l − a/2 − 1}

−\binom{(b + x − 1)/2}{(b − 1)/2 + ⌊j/2⌋ − l + 1}\binom{(b + x − 1)/2 + 1}{i + l − a/2 − 1})

Therefore, we get for the right lower corner of the matrix

M_{2i−1,2j−1} + M_{2i,2j−1} + M_{2i−1,2j} + M_{2i,2j}

= ∑_{l=1}^{⌈(a+b−1)/4⌉} (\binom{(b + x − 1)/2 + 1}{(b − 1)/2 + i − l + 1}\binom{(b + x − 1)/2 + 1}{j + l − a/2 − 1}

−\binom{(b + x − 1)/2 + 1}{(b − 1)/2 + j − l + 1}\binom{(b + x − 1)/2 + 1}{i + l − a/2 − 1}) , (8)

where i, j = 1, …, a/2.

This is almost a block matrix, only the first row and column spoil the picture.

Example (a = 8, b = 3, c = 7):

( \begin{matrix}0 & 0 & 1 & 5 & 0 & 0 & −1 & −5 \\ 0 & 0 & 3 & 12 & 0 & 0 & 0 & 0 \\ −1 & −3 & 0 & 9 & 0 & 0 & 0 & 0 \\ −5 & −12 & −9 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & −1 & −6 & −15 \\ 0 & 0 & 0 & 0 & 1 & 0 & −9 & −18 \\ 1 & 0 & 0 & 0 & 6 & 9 & 0 & −9 \\ 5 & 0 & 0 & 0 & 15 & 18 & 9 & 0 \end{matrix} )

If (a/2) is even, the right lower corner is an (a/2) × (a/2)–matrix with non-zero determinant, as we will see later, thus, we can use the last a/2 rows to annihilate the second half of the first row. This potentially changes the entry 0 in position (1, 1), but leaves everything else unchanged. We can use the same linear combination on the last a/2 columns to annihilate the second half of the first column. The resulting matrix is again skew–symmetric which means that the entry (1, 1) has returned to the value 0. Since simultaneous row and column manipulations of this kind leave the Pfaffian unchanged, it remains to find out the Pfaffian of the right lower corner (a/2 × a/2) and the Pfaffian of the left upper corner (a/2 × a/2).

The right lower block is given by Equation (8). This corresponds exactly to the first case of the ordinary enumeration of self–complementary plane partitions in Lemma 8. Therefore, the Pfaffian of this block is SC(a/2, (b + 1)/2, (c + 1)/2) (which is non-zero as claimed).

The left upper a/2 × a/2 block (including the first row and column) is

( \begin{matrix}0 & M_{1,2j+1} − M_{1,2j} \\ M_{2i+1,1} − M_{2i,1} & \begin{aligned}⌊ & \frac{a+b−1}{4} & ⌋ \\ & ∑ \\ & l=1 \end{aligned} (\binom{(b+x+1)/2}{(b−1)/2+i−l+1}\binom{(b+x+1)/2}{j+l−a/2} − \binom{(b+x+1)/2}{(b−1)/2+j−l+1}\binom{(b+x+1)/2}{i+l−a/2}) \end{matrix} ) ,

where i, j run from 0 to a/2 − 1 and

M_{2i+1,1} − M_{2i,1} = ∑_{l=1}^{⌊(a+b−1)/4⌋} (\binom{(b+x+1)/2}{(b−1)/2+i−l+1}\binom{(b+x−1)/2}{l−a/2} − \binom{(b+x−1)/2}{(b−1)/2−l+1}\binom{(b+x+1)/2}{i+l−a/2})

M_{1,2j+1} − M_{1,2j} = ∑_{l=1}^{⌊(a+b−1)/4⌋} (\binom{(b+x−1)/2}{(b−1)/2−l+1}\binom{(b+x+1)/2}{j+l−a/2} − \binom{(b+x+1)/2}{(b−1)/2+j−l+1}\binom{(b+x−1)/2}{l−a/2}) .

Note that the exceptional row and column almost fit the general pattern. We just have sometimes (b+x−1)/2 instead of (b+x+1)/2. Replace row(i) with row(i)−row(i−1) for i = 1, 2, …, a/2 − 1 in that order. Then do the same thing for the columns. In the resulting matrix all occurrences of (b + x + 1)/2 have been replaced with (b + x − 1)/2.

After shifting the indices by one, we get

\begin{aligned}⌊ & \frac{a+b−1}{4} & ⌋ \\ & ∑ \\ & l=1 \end{aligned} (\binom{(b+x−1)/2}{(b−1)/2+i−l}\binom{(b+x−1)/2}{j+l−a/2−1} − \binom{(b+x−1)/2}{(b−1)/2+j−l}\binom{(b+x−1)/2}{i+l−a/2−1}) ,

for i, j = 1, …, a/2.

The Pfaffian of this matrix can easily be identified as SC(\frac{a}{2}, \frac{b−1}{2}, \frac{c−1}{2}) by Lemma 8. Using Theorem 1, we obtain for the (−1)–enumeration

SC(\frac{a}{2}, \frac{b+1}{2}, \frac{c+1}{2})SC(\frac{a}{2}, \frac{b−1}{2}, \frac{c−1}{2}) = SC (\frac{a}{2}, \frac{b+1}{2}, \frac{c−1}{2}) SC (\frac{a}{2}, \frac{b−1}{2}, \frac{c+1}{2}) ,

which proves the main theorem in this case.

If (a/2) is odd, we move the first row and column to the (a/2)th place (which does not change the sign). Now we have an (a−2)/2×(a−2)/2–block matrix in the left upper corner which has non-zero determinant and thus can be used to annihilate the first half of the exceptional row and column similar to the previous case. By Equation (7) and Lemma 8 the Pfaffian of the left upper block is clearly SC((a−2)/2, (b+1)/2, (c+1)/2).

For the right lower (a + 2)/2 × (a + 2)/2–block, note that the relevant half of the exceptional column is

M_{2i−1,1} + M_{2i,1} = ∑_{l=1}^{⌈(a+b−1)/4⌉} (\binom{(b + x + 1)/2}{(b + 1)/2 + i − l}\binom{(b + x − 1)/2}{l − a/2 − 1}

−\binom{(b + x − 1)/2}{(b + 1)/2 − l}\binom{(b + x + 1)/2}{i + l − a/2 − 1}) .

We use again row and column operations of the type row(i)−row(i−1). This changes all occurrences of (b + x + 1)/2 to (b + x − 1)/2 and the extra row and column now fit the pattern in Equation (8) with i, j = 0. After shifting i, j to i − 1, j − 1, we identify this Pfaffian as SC((a + 2)/2, (b − 1)/2, (c − 1)/2). Again, by Theorem 1, the product of the two terms is exactly SC(a/2, (b − 1)/2, (c + 1)/2)SC(a/2, (b + 1)/2, (c − 1)/2) as claimed in the theorem.

Case a even, x odd, b, c odd We start again from the first case of Lemma 7 and have to find the Pfaffian of the matrix

M_{ij} = ∑_{k=1}^{\frac{a+b−1}{2}}(−1)^{(i+j)(k+1)} (\left[{b + x \atop b + i − k}\right]_{−1}\left[{b + x \atop j + k − a − 1}\right]_{−1}

− \left[{b + x \atop b + j − k}\right]_{−1}\left[{b + x \atop i + k − a − 1}\right]_{−1}) , 1 ≤ i, j ≤ a.

In this case, we can simply reorder the rows and columns of the matrix so that even indices come before odd indices. This introduces a sign that again cancels with (−1)^{a(a−2)/8}.

We have

M_{2i,2j−1} = ∑_{k=1}^{\frac{a+b−1}{2}}(−1)^{k+1} (\left[{b + x \atop b + 2i − k}\right]_{−1}\left[{b + x \atop 2j − 2 + k − a}\right]_{−1}

− \left[{b + x \atop b + 2j − 1 − k}\right]_{−1}\left[{b + x \atop 2i + k − a − 1}\right]_{−1}) .

Since b + x is even and either b + 2i − k or 2j − 2 + k − a has to be zero, the first product is always zero. The analogous argument for the second product gives M_{2i,2j−1} = M_{2j,2i−1} = 0.

Therefore, we have to evaluate (Pf_{1≤i,j≤a/2} M_{2i,2j})(Pf_{1≤i,j≤a/2} M_{2i−1,2j−1}) which is clearly zero for a/2 odd.

Now for a/2 even we have to evaluate the two Pfaffians. Firstly, we substitute k = 2l − 1 to obtain for the left upper block:

M_{2i,2j} = ∑_{l=1}^{⌈(a+b−1)/4⌉} (\binom{(b + x)/2}{(b + 1)/2 + i − l}\binom{(b + x)/2}{j − a/2 + l − 1}

−\binom{(b + x)/2}{(b + 1)/2 + j − l}\binom{(b + x)/2}{i − a/2 + l − 1}) , 1 ≤ i, j ≤ a/2.

We can again identify the Pfaffian of this matrix as an ordinary enumeration of selfcomplementary plane partitions by Lemma 8, namely SC(a/2, (b+1)/2, (c−1)/2) (here, (c − 1)/2 − (b + 1)/2 = (c − b)/2 − 1 which is still positive because x = (c − b)/2 is odd).

Substituting k = 2l, we obtain for the right lower block:

M_{2i−1,2j−1} = ∑_{l=1}^{⌊(a+b−1)/4⌋} (\binom{(b + x)/2}{(b − 1)/2 + i − l}\binom{(b + x)/2}{j − 1 + l − a/2}

−\binom{(b + x)/2}{(b − 1)/2 + j − l}\binom{(b + x)/2}{i − 1 + l − a/2}) , 1 ≤ i, j ≤ a/2.

By Lemma 8 this is exactly SC(a/2, (b − 1)/2, (c + 1)/2). The product is SC(a/2, (b + 1)/2, (c − 1)/2)SC(a/2, (b − 1)/2, (c + 1)/2) as claimed in the theorem.

Case: a odd, x even, b, c even According to the second case of Lemma 7 we have to evaluate Pf_{1≤i,j≤a+1} M_{ij} for

M = ( \begin{matrix}∑_{k=1}^{\frac{a+b−1}{2}}(T_{ik}T_{j,a+b+1−k} − T_{jk}T_{i,a+b+1−k}) & T_{i, \frac{a+b+1}{2}} \\ −T_{j, \frac{a+b+1}{2}} & 0 \end{matrix} ) ,

where T_{ij} = (−1)^{(x−i)(j−1)}\left[{b+x \atop b+i−j}\right]_{−1} (and x = (c − b)/2).

We reorder rows and columns so that the even ones come before the odd ones. This introduces a sign (−1)^{(a+1)(a+3)/8} and gives almost a block matrix because for i ≠ \frac{a+1}{2} we have

M_{2i,2j−1} = ∑(−1)^{k+1} (\left[{b + x \atop b + 2i − k}\right]_{−1}\left[{b + x \atop 2j + k − a − 2}\right]_{−1}

− \left[{b + x \atop b + 2j − k}\right]_{−1}\left[{b + x \atop 2i + k − a − 2}\right]_{−1}) .

Since b + x is even and either b + 2i − k or 2j + k − a − 2 is odd, we get M_{2i,2j−1} = 0. Now we look at the exceptional row:

M_{a+1,2j} = −T_{2j, \frac{a+b+1}{2}} = −\left[{b + x \atop b + 2j − \frac{a+b+1}{2}}\right]_{−1}

= \begin{cases}0 & for \frac{a+b+1}{2} odd \\ −\binom{(b + x)/2}{b/2 + j − \frac{a+b+1}{4}} & for \frac{a+b+1}{2} even. \end{cases}  (9)

M_{a+1,2j−1} = −T_{2j−1, \frac{a+b+1}{2}} = −(−1)^{\frac{a+b+1}{2} −1}\left[{b + x \atop b + 2j − 1 − \frac{a+b+1}{2}}\right]_{−1}

= \begin{cases}−\binom{(b + x)/2}{b/2 + j − 1 − \frac{a+b−1}{4}} & for \frac{a+b+1}{2} odd \\ 0 & for \frac{a+b+1}{2} even. \end{cases}  (10)

Therefore, in the subcase \frac{a+b+1}{2} even, we have a block matrix composed of two \frac{a+1}{2} × \frac{a+1}{2}–blocks. The Pfaffian is clearly zero if \frac{a+1}{2} is odd which proves the theorem in this case.

If \frac{a+1}{2} is even, we have two blocks.

The left upper \frac{a+1}{2} × \frac{a+1}{2}–block:

M_{2i,2j} = ∑_{k=1}^{\frac{a+b−1}{2}} (\left[{b + x \atop b + 2i − k}\right]_{−1}\left[{b + x \atop 2j + k − a − 1}\right]_{−1}

− \left[{b + x \atop b + 2j − k}\right]_{−1}\left[{b + x \atop 2i + k − a − 1}\right]_{−1})

= \begin{aligned}⌊ & \frac{a+b−1}{4} & ⌋ \\ & ∑ \\ & l=1 \end{aligned} (\binom{\frac{b+x}{2}}{\frac{b}{2} + i − l}\binom{\frac{b+x}{2}}{j + l − \frac{a+1}{2}} − \binom{\frac{b+x}{2}}{\frac{b}{2} + j − l}\binom{\frac{b+x}{2}}{i + l − \frac{a+1}{2}}) , (11)

for i, j ≠ \frac{a+1}{2}.

We can use Equation (9) and Lemma 8 to see that the left upper Pfaffian is exactly (−1)^{(a−3)/4}SC(\frac{a−1}{2}, \frac{b}{2}, \frac{c}{2}) (which is non-zero because b/2 is even).

The right lower block looks like

M_{2i−1,2j−1} = \begin{aligned}⌊ & \frac{a+b+1}{4} & ⌋ \\ & ∑ \\ & l=1 \end{aligned} (\binom{\frac{b+x}{2}}{\frac{b}{2} + i − l}\binom{\frac{b+x}{2}}{j − \frac{a+1}{2} − 1 + l}

− \binom{\frac{b+x}{2}}{\frac{b}{2} + j − l}\binom{\frac{b+x}{2}}{i − \frac{a+1}{2} − 1 + l}) , (12)

which is SC(\frac{a+1}{2}, \frac{b}{2}, \frac{c}{2}). It can easily be checked that the signs cancel and the product of the two terms is

exactly as claimed in the theorem.

Now we look at the subcase \frac{a+b+1}{2} odd. Equations (9) and (10) show that we have a block matrix with a left upper block of

size \frac{a−1}{2} and a right lower block of size \frac{a+3}{2}. Therefore, the Pfaffian is zero, if \frac{a−1}{2} is odd, in accordance with the claim in the theorem.

If \frac{a−1}{2} is even, the left upper block consists exactly of the entries in Equation (11). Lemma 8 identifies this Pfaffian as SC((a − 1)/2, b/2, c/2). The right lower block is given by Equation (12) together with Equation (10). We move the exceptional row and column from the first to the last place which gives a sign change. By Lemma 8, the Pfaffian of this matrix is (−1)^{(a−1)/4}SC((a + 1)/2, b/2, c/2). The signs cancel and the product of the two sub-Pfaffians is exactly as claimed in the theorem.

Case: a odd, x odd, b, c even We start again with Pf_{1≤i,j≤a+1} M in the second case of Lemma 7. For i, j < a + 1, we have

M_{ij} = ∑_{k=1}^{\frac{a+b−1}{2}}(−1)^{(k+1)(i+j)} (\binom{(b + x − 1)/2}{⌊(b + i − k)/2⌋}\binom{(b + x − 1)/2}{⌊(k + j − a − 1)/2⌋}

− \binom{(b + x − 1)/2}{⌊(b + j − k)/2⌋}\binom{(b + x − 1)/2}{⌊(k + i − a − 1)/2⌋}) .

This is almost identical to the case of a and x even, therefore we proceed similarly and split the sums for even and odd k.

M_{ij} = \begin{matrix}⌊ & \frac{a+b−1}{4} & ⌋ \\ & ∑ \\ & l=1 \end{matrix}(−1)^{i+j} (\binom{(b + x − 1)/2}{b/2 + ⌊i/2⌋ − l}\binom{(b + x − 1)/2}{l + ⌊j/2⌋ − (a + 1)/2}

− \binom{(b + x − 1)/2}{b/2 + ⌊j/2⌋ − l}\binom{(b + x − 1)/2}{l + ⌊i/2⌋ − (a + 1)/2})

+ \begin{aligned}⌈ & \frac{a+b−1}{4} & ⌉ \\ & ∑ \\ & l=1 \end{aligned} (\binom{(b + x − 1)/2}{b/2 + ⌊(i + 1)/2⌋ − l}\binom{(b + x − 1)/2}{l + ⌊(j − 1)/2⌋ − (a + 1)/2}

− \binom{(b + x − 1)/2}{b/2 + ⌊(j + 1)/2⌋ − l}\binom{(b + x − 1)/2}{l + ⌊(i − 1)/2⌋ − (a + 1)/2}) .

The extra row is

M_{a+1,j} = −T_{j,(a+b+1)/2} = −(−1)^{(j−1)((a+b−1)/2)}\binom{(b + x − 1)/2}{b/2 + ⌊(j − (a + b + 1)/2)/2⌋}.

Now we perform the following row and column operations: Replace the rows with row(1), row(2i + 1) + row(2i) (for i = 1, …, \frac{a−1}{2}), row(a + 1), row(2i) − row(2i − 1) (for i = 1, …, \frac{a−1}{2}). Then do the same thing for the columns. This introduces a sign of (−1)^{(a−1)(a+5)/8}. All the four (a + 1)/2 × (a + 1)/2–blocks of the new matrix have exceptional first rows and columns. We have

M_{2i+1,j} + M_{2i,j} = \begin{aligned}⌈ & \frac{a+b−1}{4} & ⌉ \\ & ∑ \\ & l=1 \end{aligned} (\binom{(b + x + 1)/2}{b/2 + i + 1 − l}\binom{(b + x − 1)/2}{l + ⌊(j − 1)/2⌋ − (a + 1)/2}

− \binom{(b + x − 1)/2}{b/2 + ⌊(j + 1)/2⌋ − l}\binom{(b + x + 1)/2}{l + i − (a + 1)/2}) .

Therefore, the right upper block (M_{2i+1,2j} +M_{2i,2j} −M_{2i+1,2j−1} −M_{2i,2j−1}) apart from its first row and first column is 0.

The left upper block without its first row and column is given by

M_{2i+1,2j+1} + M_{2i,2j+1} + M_{2i+1,2j} + M_{2i,2j}

= \begin{aligned}⌈ & \frac{a+b−1}{4} & ⌉ \\ & ∑ \\ & l=1 \end{aligned} (\binom{(b + x + 1)/2}{b/2 + i + 1 − l}\binom{(b + x + 1)/2}{l + j − (a + 1)/2}

− \binom{(b + x + 1)/2}{b/2 + j + 1 − l}\binom{(b + x + 1)/2}{l + i − (a + 1)/2}) , (13)

for i, j = 1, …, (a − 1)/2.

The first column of the left upper block is given by

18 THERESIA EISENKÖLBL

M_{2i+1,1} + M_{2i,1} = \begin{aligned}⌈ & \frac{a+b−1}{4} & ⌉ \\ & ∑ \\ & l=1 \end{aligned} (\binom{(b + x + 1)/2}{b/2 + i + 1 − l}\binom{(b + x − 1)/2}{l − (a + 1)/2}

− \binom{(b + x − 1)/2}{b/2 + 1 − l}\binom{(b + x + 1)/2}{l + i − (a + 1)/2}) . (14)

Furthermore, we compute

M_{2i,j} − M_{2i−1,j} = \begin{matrix}⌊ & \frac{a+b−1}{4} & ⌋ \\ & ∑ \\ & l=1 \end{matrix}(−1)^{j} (\binom{(b + x + 1)/2}{b/2 + i − l}\binom{(b + x − 1)/2}{l + ⌊j/2⌋ − (a + 1)/2}

− \binom{(b + x − 1)/2}{b/2 + ⌊j/2⌋ − l}\binom{(b + x + 1)/2}{l + i − (a + 1)/2}) .

The right lower block without its first row and column is given by M_{2i,2j} − M_{2i−1,2j} − M_{2i,2j−1} + M_{2i−1,2j−1}

= \begin{aligned}⌊ & \frac{a+b−1}{4} & ⌋ \\ & ∑ \\ & l=1 \end{aligned} (\binom{(b + x + 1)/2}{b/2 + i − l}\binom{(b + x + 1)/2}{l + j − (a + 1)/2}

− \binom{(b + x + 1)/2}{b/2 + j − l}\binom{(b + x + 1)/2}{l + i − (a + 1)/2}) (15)

for i, j = 1, … (a − 1)/2.

The second half of the first column is given by

M_{2i,1} − M_{2i−1,1} = − \begin{aligned}⌊ & \frac{a+b−1}{4} & ⌋ \\ & ∑ \\ & l=1 \end{aligned} (\binom{(b + x + 1)/2}{b/2 + i − l}\binom{(b + x − 1)/2}{l − (a + 1)/2}

− \binom{(b + x − 1)/2}{b/2 − l}\binom{(b + x + 1)/2}{l + i − (a + 1)/2}) . (16)

Finally, the other exceptional row and column are given by M_{a+1,2j} − M_{a+1,2j−1} = −T_{2j,(a+b+1)/2} + T_{2j−1,(a+b+1)/2}

= −(−1)^{(a+b+1)/2−1}\left[{b + x \atop b + 2j − (a + b + 1)/2}\right]_{−1} + \left[{b + x \atop b + 2j − 1 − (a + b + 1)/2}\right]_{−1}

= \begin{cases}\binom{(b+x+1)/2}{b/2+j−(a+b+1)/4} & for \frac{a+b+1}{2} even, \\ 0 & for \frac{a+b+1}{2} odd. \end{cases} (17)

and M_{2i+1,a+1} + M_{2i,a+1} = T_{2i+1,(a+b+1)/2} + T_{2i,(a+b+1)/2}

= \left[{b + x \atop b + 2i + 1 − (a + b + 1)/2}\right]_{−1} + (−1)^{(a+b+1)/2−1}\left[{b + x \atop b + 2i − (a + b + 1)/2}\right]_{−1}

= \begin{cases}0 & for \frac{a+b+1}{2} even, \\ \binom{(b+x+1)/2}{b/2+i−(a+b−1)/4} & for \frac{a+b+1}{2} odd. \end{cases} (18)

Now we look at the subcase \frac{a+b+1}{2} even. Then M_{2i+1,a+1} + M_{2i,a+1} = 0. If (a+1)/2 is even, we will see that the right lower (a + 1)/2 × (a + 1)/2–matrix has a non-zero determinant and we can treat the matrix as a block matrix despite the first exceptional row.

By simply subtracting each row and column from its successor, we can change the left upper corner so that all (b+x+1)/2 become (b+x−1)/2. The first row and column now fit in with i, j = 0 and an index shift by one gives the (a + 1)/2 × (a + 1)/2–matrix:

\begin{aligned}⌈ & \frac{a+b−1}{4} & ⌉ \\ & ∑ \\ & l=1 \end{aligned} (\binom{(b + x − 1)/2}{b/2 + i − l}\binom{(b + x − 1)/2}{l + j − 1 − (a + 1)/2}

− \binom{(b + x − 1)/2}{b/2 + j − l}\binom{(b + x − 1)/2}{l + i − 1 − (a + 1)/2}) .

By Lemma 8, the left upper block has Pfaffian SC((a + 1)/2, b/2, (c − 2)/2).

The right lower block is given by (15) and (17) with Pfaffian (−1)^{(a−3)/4}SC((a − 1)/2, b/2, (c + 2)/2) which is not zero.

The signs cancel again, and by Theorem 1, we have

SC((a + 1)/2, b/2, (c − 2)/2)SC((a − 1)/2, b/2, (c + 2)/2) = SC((a − 1)/2, b/2, c/2)SC((a + 1)/2, b/2, c/2)

as claimed in our theorem.

If (a+1)/2 is odd, we multiply the first row and column by (−1) and move them to the place of the other special row and column, these are moved to the last place. These operations change the sign. The left upper (a − 1)/2 × (a − 1)/2–matrix has a non-zero determinant and thus, we can treat the matrix as a block matrix.

By Lemma 8, the left upper block given in (13) has Pfaffian

SC((a − 1)/2, (b + 2)/2, c/2).

In the same way as in previous cases, we can use the row and column operations of type row(i) − row(i − 1) to obtain a right lower (a + 1)/2 × (a + 1)/2–block given by

\begin{aligned}⌊ & \frac{a+b−1}{4} & ⌋ \\ & ∑ \\ & l=1 \end{aligned} (\binom{(b + x − 1)/2}{b/2 + i − l − 1}\binom{(b + x − 1)/2}{l + j − (a + 1)/2 − 1}

− \binom{(b + x − 1)/2}{b/2 + j − l − 1}\binom{(b + x − 1)/2}{l + i − (a + 1)/2 − 1}) ,

for i, j = 1, …, (a + 1)/2 and the row

\binom{(b + x − 1)/2}{b/2 + j − 1 − (a + b + 1)/4}, for j = 1, …, (a + 1)/2).

(Just apply the mentioned row and column operations to (−M_{a+1,1}) = \binom{(b+x−1)/2}{b/2−(a+b+1)/4} and the expression in Equation (17).)

By Lemma 8 the Pfaffian of this matrix is −(−1)^{(a−1)/4}SC((a + 1)/2, (b − 2)/2, c/2). It can easily be checked that the signs cancel again and the product of the two

expressions is exactly as claimed in the theorem.

Now we look at the subcase \frac{a+b+1}{2} odd.

If (a+1)/2 is odd, the right lower (a − 1)/2 × (a − 1)/2–block has non-zero determinant. The second half of the (a + 1)–row is zero by equation (17). Therefore, we can eliminate the second half of the first row and column and have a block matrix. Equation 15 and Lemma 8 show that the Pfaffian of the right lower block is SC((a − 1)/2, b/2, (c + 2)/2).

The left upper (a + 3)/2 × (a + 3)/2–block is given by the expressions in (13), (14),

(18) and M_{a+1,1} = −\binom{(b+x−1)/2}{b/2−(a+b−1)/4}.

The row and column operations row(i) − row(i − 1) again replace all occurrences of (b + x + 1)/2 with (b + x − 1)/2. The first row and column fits in with i, j = 0 (also for the a + 1–entry), and we get an (a + 3)/2 × (a + 3)/2–block starting with

∑_{l=1}^{⌈(a+b−1)/4⌉} (\binom{(b + x − 1)/2}{b/2 + i − l}\binom{(b + x − 1)/2}{l + j − 1 − (a + 1)/2}

− \binom{(b + x − 1)/2}{b/2 + j − l}\binom{(b + x − 1)/2}{l + i − 1 − (a + 1)/2}) ,

for i, j = 1, …, (a + 1)/2 while the entries of the extra row are

−\binom{(b + x − 1)/2}{b/2 + j − (a + b + 3)/4}.

Lemma 8 immediately shows that the Pfaffian of this matrix is

(−1)^{(a−1)/4}SC((a + 1)/2, b/2, (c − 2)/2).

The signs cancel and the product

SC((a − 1)/2, b/2, (c + 2)/2)SC((a + 1)/2, b/2, (c − 2)/2)

is equal to the expression claimed in the theorem.

If (a+1)/2 is even, we start by moving the first row and column after the other special row and column.

Now the left upper (a + 1)/2 × (a + 1)/2–block (given by (13) and (18)) has non-zero determinant and can be used to annihilate the first half of the former first row and column. By Lemma 8 the Pfaffian of the left upper block is

(−1)^{(a−3)/4}SC((a − 1)/2, (b + 2)/2, c/2).

The right lower block is given by (16) and (15). We multiply the first row and column by (−1) and use row and column operations similar to the previous cases to obtain the (a + 1)/2 × (a + 1)/2–block

\begin{aligned}⌊ & \frac{a+b−1}{4} & ⌋ \\ & ∑ \\ & l=1 \end{aligned} (\binom{(b + x − 1)/2}{b/2 + i − l − 1}\binom{(b + x − 1)/2}{l + j − 1 − (a + 1)/2}

− \binom{(b + x − 1)/2}{b/2 + j − l − 1}\binom{(b + x − 1)/2}{l + i − 1 − (a + 1)/2}) .

By Lemma 8, this is SC((a + 1)/2, (b − 2)/2, c/2). The signs cancel again and the product SC((a − 1)/2, (b + 2)/2, c/2)SC((a + 1)/2, (b − 2)/2, c/2) is easily seen to be equal to the expression in the theorem.

This case concludes the proof of the theorem. □

Remark. In Equation (6) we see that whereever x occurs, there is actually the expression (b+x−1)/2. Now replace all occurences by four different variables in the following way:

M_{ij}(m_{1}, m_{2}, n_{1}, n_{2}, a, b)

= ∑_{l=1}^{⌊(a+b−1)/4)⌋}(−1)^{i+j} (\binom{n_{1}}{(b − 1)/2 + ⌊(i − 1)/2⌋ − l + 1}\binom{m_{1}}{−a/2 + ⌊(j − 1)/2⌋ + l}

− \binom{n_{1}}{(b − 1)/2 + ⌊(j − 1)/2⌋ − l + 1}\binom{m_{1}}{−a/2 + ⌊(i − 1)/2⌋ + l})

+ ∑_{l=1}^{⌈(a+b−1)/4⌉} (\binom{n_{2}}{(b − 1)/2 + ⌊i/2⌋ − l + 1}\binom{m_{2}}{−a/2 + ⌊j/2⌋ + l − 1}

− \binom{n_{2}}{(b − 1)/2 + ⌊j/2⌋ − l + 1}\binom{m_{2}}{−a/2 + ⌊i/2⌋ + l − 1}) ,

where i, j = 1, …, a.

Experimentally, the Pfaffian of this matrix is a product of linear factors each involving only one of the four variables. Each factor corresponds to one of the B(r, s, t)–factor obtained by applying Theorem 1 to Theorem 2.

References

[1] G. E. Andrews, Plane partitions (II): The equivalence of the Bender–Knuth and the MacMahon conjectures, Pacific J. Math., 72 (1977), 283–291.

[2] G. E. Andrews, Plane partitions (I): The MacMahon conjecture, Adv. in Math. Suppl. Studies, 1 (1978), 131–150.

[3] G. E. Andrews, Plane partitions V: The t.s.s.c.p.p. conjecture, J. Combin. Theory Ser. A, 66 (1994), 28–39.

[4] T. Eisenkölbl, (−1)-enumeration of plane partitions with complementation symmetry, Adv. in Appl. Math. 30, (2003), no. 1-2, 53–95, arXiv:math.CO/0011175.

[5] I.M. Gessel, X. Viennot, Determinant, paths and plane partitions, Preprint, (1989).

[6] M. Ishikawa and M. Wakayama, Minor summation formula of Pfaffians, Linear and Multilinear Algebra 39 (1995), 285–305.

[7] G. Kuperberg, An exploration of the permanent-determinant method, Electron. J. Combin. 5, (1998), #R46, arXiv:math.CO/9810091.

[8] G. Kuperberg, Symmetries of plane partitions and the permanent determinant method, J. Combin. Theory Ser. A 68 (1994), 115–151.

[9] B. Lindström, On the vector representations of induced matroids, Bull. London Math. Soc. 5 (1973), 85–90.

[10] P.A. MacMahon, Combinatory Analysis, vol. 2, Cambridge University Press, (1916); reprinted by Chelsea, New York, (1960).

[11] W. H. Mills, D. H. Robbins and H. Rumsey, Proof of the Macdonald conjecture, Inventiones Math. 66, (1982), 73–87.

[12] S. Okada, Applications of minor summation formulas to rectangular-shaped representations of classical groups, J. Algebra, 205, (1998), 337–367.

[13] R. P. Stanley, Enumerative Combinatorics, Vol. 1, Wadsworth & Brooks/Cole, Pacific Grove, California, (1986).

[14] R.P. Stanley, Symmetries of plane partitions, J. Combin. Theory Ser A 43 (1986), 103–113; Erratum 44 (1987), 310.

[15] J.R. Stembridge, Some hidden relations involving the ten symmetry classes of plane partitions, J. Combin. Theory Ser. A 68 (1994), 372–409.

[16] J.R. Stembridge, On minuscule representations, plane partitions and involutions in complex Lie groups, Duke Math. J. 73 (1994), 469–490.

[17] J. R. Stembridge, The enumeration of totally symmetric plane partitions, Adv. in Math. 111 (1995), 227–245.