RHOMBUS TILINGS OF A HEXAGON WITH TWO TRIANGLES MISSING ON THE SYMMETRY AXIS

THERESIA EISENKÖLBL

Institut für Mathematik der Universität Wien, Strudlhofgasse 4, A-1090 Wien, Austria.

                     E-mail: Theresia.Eisenkoelbl@univie.ac.at

Submitted: May 16, 1999; Accepted: July 27, 1999.

Abstract. We compute the number of rhombus tilings of a hexagon with sides n, n, N, n, n, N, where two triangles on the symmetry axis touching in one vertex are removed. The case of the common vertex being the center of the hexagon solves a problem posed by Propp.

1. Introduction

The interest in rhombus tilings has emerged from the enumeration of plane partitions in a given box. The connection comes from looking at the stacks of cubes of a plane partition and projecting the picture to the plane. Then the box becomes a hexagon, where opposite sides are equal, and the cubes become a rhombus tiling of the hexagon where the rhombi consist of two equilateral triangles (cf. [2]). The number of plane partitions in a given box was first computed by MacMahon [7, Sec. 429, q → 1, proof in Sec. 494]. Therefore:

The number of all rhombus tilings of a hexagon with sides a, b, c, a, b, c equals

∏_{i=1}^{a} ∏_{j=1}^{b} ∏_{k=1}^{c} \frac{i + j + k − 1}{i + j + k − 2} = ∏_{i=1}^{a} \frac{(c + i)_{b}}{(i)_{b}} .  (1)

(The symmetric first expression is due to Macdonald.)

In [8], Propp proposed several problems regarding “incomplete” hexagons, i.e., hexagons, where certain triangles are missing. In particular, Problem 4 of [8] asks for a formula for the number of rhombus tilings of a regular hexagon, where two of the six central triangles are missing. We treat the case of the two triangles lying on the symmetry axis and touching in one vertex (see Figure 1). The other case has been solved in [3]. We prove the following two theorems.

1991 Mathematics Subject Classification. Primary 05A15; Secondary 05A19 05B45 33C20 52C20.

Key words and phrases. lozenge tilings, rhombus tilings, plane partitions, determinants, nonintersecting lattice paths.

Theorem 1. The number of rhombus tilings of a hexagon with sides n, n, 2m, n, n, 2m and two missing triangles on the horizontal symmetry axis sharing the (s+1)–th vertex on the axis (counted from the left, see Figure 1) equals

\frac{(2m − 1)\binom{2m−2}{m−1}\binom{2n−2s}{n−s}\binom{2s}{s}}{\binom{2m+2n}{m+n}} ∏_{i=1}^{n} \frac{(2m + i)_{n}}{(i)_{n}} .

Theorem 2. The number of rhombus tilings of a hexagon with sides n, n, 2m + 1, n, n, 2m + 1 and two missing triangles on the symmetry axis sharing the s–th vertex on the axis equals

\frac{(2m + 1)\binom{2m}{m}\binom{2n−2s}{n−s}\binom{2s−2}{s−1}}{\binom{2m+2n}{m+n}} ∏_{i=1}^{n} \frac{(2m + i + 1)_{n}}{(i)_{n}} .

The following corollary is easily derived using Stirling’s approximation formula.

Corollary. The proportion of rhombus tilings of a hexagon with sides αt, αt, βt, αt, αt, βt and two missing triangles on the horizontal symmetry axis touching the (γt)-th vertex on the axis in the number of all rhombus tilings of the hexagon with sidelengths αt, αt, βt, αt, αt, βt (given in (1)) is asymptotically equal to

\frac{1}{4π} \sqrt{\frac{β(2α + β)}{γ(α − γ)}}.

This expression can attain arbitrary large values if γ is close to α or 0, i.e. the missing triangles lie near the border of the hexagon. The expression equals \frac{\sqrt{3}}{2π} (which is approximately 0.28) for α = β = 2γ, which corresponds to the case of a regular hexagon with two missing triangles touching the center. In comparison, in the other case of Problem 4 of [8], the case of a fixed rhombus on the symmetry axis, the analogous proportion must always be smaller than 1 and equals approximately \frac{1}{3} if the central rhombus is missing (see [3]).

The rest of the paper is devoted to the proof of Theorems 1 and 2. The main ingredients are the matchings factorization theorem by M. Ciucu [1], nonintersecting lattice paths, and two determinant evaluations, the latter constituting the most difficult part of the proof. An outline of the proof is given in the next section. The details are filled in in the subsequent sections.

2. Outline of the proofs of Theorems 1 and 2

Outline of the proof of Theorem 1:

s + 1 2 m {  {  } n  n

Figure 1. A hexagon with sides n, n, 2m, n, n, 2m and missing triangles in position s + 1, where m = 2, n = 3, s = 2, and a rhombus tiling.

Figure 2. We place a dot in every bounded region and connect dots corresponding to adjacent regions. The dual graph of the six triangles on the left is the hexagon on the right.

Step 1: It suffices to compute the number of rhombus tilings of two regions R^{+} and R^{−} which are roughly the upper and the lower half of the original hexagon (see Section 3).

We use the fact that there is a bijection between rhombus tilings of the hexagon and perfect matchings of the hexagon’s ‘dual’ graph G (see Figure 2 for the construction of the dual graph and Figure 3a for the correspondence between tilings and matchings).

The graph G has reflective symmetry, so the matchings factorization theorem by M. Ciucu [1] is applicable (see Lemma 3). This theorem expresses the number of perfect matchings of a graph as a power of two times the numbers of perfect matchings of two smaller graphs G^{+} and G^{−} (see Lemma 4 and Figure 3b), which are roughly the two halves of the original graph G. The remaining task is to count the numbers of perfect matchings of G^{+} and G^{−}.

We use again the correspondence between the rhombus tilings of a region of triangles and the perfect matchings of the dual graph and reduce the problem to counting the rhombus tilings of two regions R^{+} and R^{−} (see Figure 4).

Step 2: The numbers of rhombus tilings of R^{+} and R^{−} are certain determinants (see Sections 4 and 5).

The rhombus tilings are in bijection with certain families of nonintersecting lattice paths (see Figures 5 and 6). Application of the main result of nonintersecting lattice paths expresses the desired numbers as determinants (see equations (2) and (4)).

Step 3: Evaluation of the determinant corresponding to R^{+} (see Section 4).

The determinant corresponding to R^{+} is evaluated using a lemma by Krattenthaler (see equation (3) and Lemma 5).

Step 4: Evaluation of the determinant corresponding to R^{−} (see Section 5).

We take factors out of the determinant, so that we obtain a determinant whose entries are polynomials in m (see the proof of Lemma 6). This determinant is evaluated by using the “identification of factors” method, as explained in [5, Sec. 2]. The corresponding details are the subject of Sections 6 – 8.

Step 5: A combination of the previous steps proves Theorem 1.

We substitute the results of Lemmas 5 and 6 in Lemma 4 and obtain the following expression for the number of rhombus tilings of our original hexagon,

2^{\binom{n}{2}−1} \frac{(H(n))^{2} (2n − 2s − 1)!!(2s − 1)!!}{H(2n)(n − s)!s!}

× ∏_{2≤i≤j≤n} (2m + 2j − i) ∏_{k=1}^{n−2} (m + k + \frac{1}{2})^{min(k,n−1−k)} ∏_{k=0}^{n} (m + k)^{min(k+1,n−k+1)}.

Here, H(n) stands for ∏_{i=0}^{n−1} i!. This can easily be transformed to the expression in Theorem 1, so the proof of Theorem 1 is complete.

The proof of Theorem 2 is given in Section 9. Analogously to the proof of Theorem 1, it is enough to count the rhombus tilings of two regions R̃^{+} and R̃^{−} (see Figure 7) which are roughly the halves of the original hexagon. This can then be reduced to the determinants already evaluated in the proof of Theorem 1.

3. Breaking the hexagon in two parts

We start the proof of Theorem 1 by forming the inner dual of the given hexagon (see Figure 2). I.e., we replace every triangle by a vertex and connect vertices corresponding to adjacent triangles (see Figure 2). Thus, we get a hexagonal graph, whose perfect matchings correspond to rhombus tilings of the original hexagon (see Figure 3a).

Now we use a theorem by M. Ciucu (see [1]) to, roughly speaking, break the hexagonal graph into two halves. Before we can state it we need a few definitions. First, let H be a graph and assign to each of its edges a number, the weight of the edge. Then the weight of a perfect matching of H is the product of all weights of edges contained in the perfect matching. The weighted enumeration M(H) is just the sum of the weights of all possible perfect matchings. If every edge has weight 1 then M(H) reduces to the number of perfect matchings.

a. The perfect matching corresponding to the rhombus tiling of Figure 1.  G^{+} G^{−} b. G^{+} and G^{−}.

Figure 3.

a. R^{+}, the upper half of the hexagon.  b. R^{−}, the lower half of the hexagon.

Figure 4.

Now we describe how to get the two halves G^{+} and G^{−} from G (see Figure 3b). Let G be a planar bipartite graph with reflective symmetry, which splits into two parts after removal of the vertices of the symmetry axis. We can clearly assume that the symmetry axis is the x-axis. Label the vertices of G on the symmetry axis a_{1}, b_{1}, a_{2}, b_{2}, …, a_{2l}, b_{2l} from left to right. Since G is bipartite, we can colour the vertices of the graph black and white subject to the conditions that a_{1} is white and no two adjacent vertices are of the same colour.

Then we delete all edges connecting white a–vertices and black b–vertices to the upper half and all edges connecting black a–vertices and white b–vertices to the lower half and we divide by two all weights of edges lying on the symmetry axis. The graph G splits into two parts G^{+} and G^{−}. Now we can state the matchings factorization theorem from [1].

Lemma 3. Let G be a planar bipartite weighted, symmetric graph, which splits into two parts after removal of the vertices of the symmetry axis. Then

M(G) = 2^{l(G)}M(G^{+})M(G^{−}),

where M(H) denotes the weighted count of perfect matchings of the graph H and G^{±} denote the upper and lower half of G as described above. 2 l(G) is the number of vertices on the symmetry axis.

We apply Lemma 3 to our hexagonal graph, exemplified in Figure 3a, with respect to the horizontal symmetry axis. In our case l(G) = n − 1. G^{+} and G^{−} are shown in Figure 3b.

Thus we get the following lemma.

Lemma 4. The number of rhombus tilings of a hexagon with sides n, n, 2m, n, n, 2m and two missing triangles on the symmetry axis sharing the (s + 1)–th vertex on the axis equals

2^{n−1}M(G^{+})M(G^{−}),

where G^{+} and G^{−} are formed by the above procedure, as exemplified in Figure 3b.

M(G^{+}) and M(G^{−}) are computed in the following sections.

4. The enumeration of matchings for the upper half

In this section we evaluate M(G^{+}). The result is stated in the following lemma.

Lemma 5.

M(G^{+}) = \frac{H(n) ∏_{2≤i≤j≤n} (2m + 2j − i)}{∏_{j=1}^{n} (2j − 2)!}.

Proof. We start by expressing M(G^{+}) as the following determinant.

M(G^{+}) = det_{1≤i,j≤n} (\binom{m + j − 1}{m − j + i}) .  (2)

To this end, we convert G^{+} back to the corresponding region of triangles, R^{+} say (see Figure 4a), so that perfect matchings of G^{+} correspond bijectively to the rhombus tilings of R^{+}. Thus, we have to count rhombus tilings of R^{+}. The next step is converting rhombus tilings to families of nonintersecting lattice paths, where nonintersecting means that different lattice paths have no common vertices. The reader should consult Figure 5, while reading the following passage. Given a rhombus tiling of R^{+}, the lattice paths start on the centers of upper left diagonal edges (lying on one of the sides of length n). They end on the lower right edges parallel to the starting edges. The paths are generated by connecting the center of the respective edge with the center of the edge lying opposite in the rhombus. This process is iterated using the new edge and the second rhombus it bounds. It terminates on the lower right boundary edges. It is obvious that paths starting at different points have no common vertices, i.e., are nonintersecting. Furthermore, an arbitrary family of nonintersecting paths from the set of the upper left edges to the set of the lower right edges lies completely inside R^{+} and can be converted back to a tiling (see Figure 5a).

Then we transform the picture to “orthogonal” paths with positive horizontal and negative vertical steps of unit length (see Figure 5b,c). Let the starting points of the paths be denoted by P_{1}, P_{2}, …, P_{n} and the end points by Q_{1}, Q_{2}, …, Q_{n}. We can easily write down the coordinates of the starting points and the end points:

P_{i} = (i − 1, i + m − 1)  for i = 1, …, n,
Q_{j} = (2j − 2, j − 1)  for j = 1, …, n.

Next we apply the main result for nonintersecting lattice paths [4, Cor.2] (see also [10, Theorem 1.2]). We state it for the enumeration of weighted nonintersecting lattice paths which we will use in later sections. The weight of a family of paths is the product of the weights of all occurring edges. If each edge is assigned a weight of 1 we have a result for the ordinary enumeration.

The theorem says that the weighted count of families of nonintersecting lattice paths, with path i running from P_{i} to Q_{i}, is the determinant of the matrix with (i, j)- entry the weight P(P_{i} → Q_{j}) of lattice paths running from P_{i} to Q_{j}, provided that every two paths P_{i} → Q_{j} and P_{k} → Q_{l} have a common vertex if i < j and k > l. It is easily checked that our sets of starting and end points meet the required conditions.

The number of lattice paths with positive horizontal and negative vertical steps from (a, b) to (c, d) equals \binom{c−a+b−d}{b−d}. Therefore, the number of families of nonintersecting lattice paths (equivalently, the number of rhombus tilings of R^{+}) is equal to the following determinant:

det_{1≤i,j≤n}(P(P_{i} → Q_{j})) = det_{1≤i,j≤n} (\binom{m + j − 1}{m − j + i}) .

This proves equation (2).

This determinant can be evaluated with the help of the following determinant identity ([6], Lemma 2.2).

det_{1≤i,j≤n} ((x_{j} + a_{n})(x_{j} + a_{n−1}) ⋯ (x_{j} + a_{i+1})(x_{j} + b_{i})(x_{j} + b_{i−1}) … (x_{j} + b_{2}))

= ∏_{1≤i<j≤n} (x_{i} − x_{j}) ∏_{2≤i≤j≤n} (b_{i} − a_{j}). (3)

a. A tiling of the upper half of the hexagon and the corresponding lattice path family.  b. The paths isolated. c. The corresponding lattice path family.

•  _{•}  _{•}  _{•}  _{•}^{•}  ^{•}  ^{•}  ^{•}  ^{•}^{•}  ^{•}  ^{•}  ^{•}  ^{•}^{•}  ^{•}  ^{•}  ^{•}  ^{•}^{•}  ^{•}  ^{•}  ^{•}  ^{•} ∘ ∘  ∘ ∘  ∘ ∘^{P_{1}}  ^{P_{2}}  ^{P_{3}}_{Q_{1}}  ^{Q_{2}}  ^{Q_{3}}

Figure 5.

Before we can apply identity (3), we have to transform the expression of equation (2) in the following way.

\begin{aligned}M(G^{+}) & = det_{1≤i,j≤n} (\binom{m + j − 1}{m − j + i}) \\ & = ∏_{i=1}^{n} (−2)^{i−1} ∏_{j=1}^{n} \frac{(m + j − 1)!}{(n + m − j)!(2j − 2)!} \end{aligned}

× det_{1≤i,j≤n} ((m + i + 1 − j)(m + i + 2 − j) … (m + n − j)

· (−j + \frac{i}{2}) (−j + \frac{i}{2} − \frac{1}{2}) … (−j + 1)) .

a. A tiling of the lower half of the hexagon and the corresponding lattice path family.  b. The paths isolated. c. The corresponding lattice path family.

•  _{•}  _{•}  _{•}  _{•}  _{•}^{•}  ^{•}  ^{•}  ^{•}  ^{•}  ^{•}^{•}  ^{•}  ^{•}  ^{•}  ^{•}  ^{•}^{•}  ^{•}  ^{•}  ^{•}  ^{•}  ^{•} ∘  ∘ ∘ ∘ ∘  ∘^{R_{1}}  ^{R_{2}} ^{R_{3}}  _{S_{1}}  ^{S_{2}}  ^{S_{3}}^{\frac{1}{2}}

Figure 6.

Now we apply the identity (3) with x_{k} = −k, a_{k} = m + k, b_{k} = \frac{k}{2} and simplify to get the claimed result.

5. The enumeration of matchings for the lower half In this section we evaluate M(G^{−}). The result is stated in the following lemma.

Lemma 6.

M(G^{−}) = \frac{2^{\binom{n−1}{2}} H(n)(2n − 2s − 1)!!(2s − 1)!!}{(2n − 2s) ∏_{i=1}^{n} (2n + 1 − 2i)!(n − s − 1)!s!}

× ∏_{k=1}^{n−2} (m + k + \frac{1}{2})^{min(k,n−1−k)} ∏_{k=0}^{n} (m + k)^{min(k+1,n−k+1)}.

Proof. We start analogously to Section 4 and convert G^{−} (exemplified in Figure 3b) back to a region R^{−} of triangles (see Figure 4b), so that the perfect matchings of G^{−} correspond bijectively to the rhombus tilings of R^{−}. However, since G^{−} contains edges on the symmetry axis of G, which, by Lemma 3, has the consequence that they count with weight \frac{1}{2} in G^{−}, we are dealing with a weighted count of the rhombus tilings of R^{−}, where rhombi such as the top-left rhombus in Figure 6a count with weight \frac{1}{2}. Again, we count the rhombus tilings of R^{−} by counting the number of nonintersecting lattice path families leading from upper left to lower right edges. The starting and end points can be easily read off Figure 6.

The starting points are (see Figure 6 and note that the missing triangles at the (s + 1)–th point of the former symmetry axis make the (s + 1)–th starting point shift a step to the South-West):

R_{i} = \begin{cases}(2i − 2, m + i − 1) & for i ≠ s + 1 \\ (2s − 1, m + s − 1) & for i = s + 1. \end{cases}

The end points are:

S_{j} = (n + j − 1, j − 1) for j = 1, …, n.

Now we apply again the main result for nonintersecting lattice paths. The matrix entries are P(R_{i} → S_{j}). We note that a positive horizontal step starting at R_{i}, i ≠ s + 1 corresponds to a rhombus of weight \frac{1}{2} in R^{−}, so paths starting with a horizontal step at R_{i}, i ≠ s + 1 are counted with weight \frac{1}{2}. Therefore, we count paths starting with a horizontal step and paths starting with a vertical step separately.

We get M(G^{−}) = det_{1≤i,j≤n} (A_{ij}), where

A_{ij} = P(R_{i} → S_{j}) = \begin{cases}\frac{1}{2}\binom{n + m − i}{m + i − j} + \binom{n + m − i}{m + i − 1 − j} & for i ≠ s + 1 \\ \binom{n + m − s}{m + s − j} & for i = s + 1. \end{cases} (4)

Since this expression also makes sense for s = 0, we can include this case in the following calculations. We take factors out of the rows of det (A_{ij}), so that the remaining entries B_{ij} are polynomials in m and get the following equation

det (A_{ij}) = \frac{(n + m − s)(s + m)}{(2n − 2s) ∏_{i=1}^{n} (2n + 1 − 2i)!} × det(B_{ij}), s = 0, …, n − 1,

where

B_{ij} = \begin{cases}(n + 2 + j − 2i)_{n−j}(i + m + 1 − j)_{j−1} (m + \frac{n}{2} + \frac{1}{2} − \frac{j}{2}) & i ≠ s + 1 \\ (n + 1 + j − 2s)_{n−j}(s + m + 1 − j)_{j−1} & i = s + 1. \end{cases} (5)

Here (a)_{n} := a(a + 1)(a + 2) … (a + n − 1) is the usual shifted factorial. The entry for i ≠ s + 1 can also be written as

B_{ij} = \frac{1}{2}(n + 1 + j − 2i)_{n−j+1}(i + m + 1 − j)_{j−1} + (n + 2 + j − 2i)_{n−j}(i + m − j)_{j}. (6)

It remains to prove the following identity

det (B_{ij}) = \frac{2^{\binom{n−1}{2}} H(n)(2n − 2s − 1)!!(2s − 1)!!}{(n − s − 1)!s!(m + s)(m + n − s)}

× ∏_{k=1}^{n−2} (m + k + \frac{1}{2})^{min(k,n−1−k)} ∏_{k=0}^{n} (m + k)^{min(k+1,n−k+1)}. (7)

Outline of the proof of equation (7): In Section 6 we prove that ∏_{k=1}^{n−2} (m + k + \frac{1}{2})^{min(k,n−1−k)} divides det (B_{ij}) as a polynomial in m. In Section 7 we prove that ∏_{k=0}^{n} (m + k)^{min(k+1,n−k+1)} divides det (B_{ij}) as a polynomial in m.

We show this with the help of linear combinations of rows and columns, which vanish, if one of the linear factors is set equal to zero.

In Section 8 we compute the degree of the determinant as a polynomial in m. It is exactly equal to the number of linear factors we have already found to divide the determinant. So we know the determinant up to a constant factor. We compute this constant in equation (16) by replacing each entry by its leading coefficient and using Vandermonde’s determinant formula.

Equation (7) follows immediately from (8), (10) and (16).

6. The “half-integral” factors of det (B_{ij})

In this section we prove the following (see equation (5) for the definition of B_{ij}).

∏_{k=1}^{n−2} (m + k + \frac{1}{2})^{min(k,n−1−k)}divides det (B_{ij}) as a polynomial in m.  (8)

We find linear combinations of columns which give zero for m = − (k + \frac{1}{2}). First, we show that the following linear combination of columns equals zero for i ≠ s + 1, 0 ≤ l ≤ k, l ≥ 2k − n + 1:

∑_{j=0}^{l} \binom{l}{j}B_{i,n+2l−2k−j}|_{m=−k− \frac{1}{2}} = 0.  (9)

In order to establish this, we break the sum in two parts according to the two summands of B_{ij} in equation (6) and convert them to hypergeometric form. The left-hand side of (9) becomes

(k − 2l + i − 1 − \frac{1}{2} − n)_{n−2k+2l}(2 − 2k + 2l − 2i + 2n)_{2k−2l}

×_{2}F_{1}[\begin{matrix}−l, −1 + 2k − 2l + 2i − 2n \\ 2k − 2l + i − k − \frac{1}{2} − n \end{matrix}; 1]

+ (1 + 2k − 2l + i − k − \frac{1}{2} − n)_{−1−2k+2l+n}(2 − 2k + 2l − 2i + 2n)_{2k−2l}

×_{2}F_{1}[\begin{matrix}−l, −1 + 2k − 2l + 2i − 2n \\ 1 + 2k − 2l + i − k − \frac{1}{2} − n \end{matrix}; 1] .

Now, Vandermonde’s summation formula

2F_{1}[\begin{matrix}a, −n \\ c \end{matrix}; 1] = \frac{(−a + c)_{n}}{(c)_{n}}

is applicable to both _{2}F_{1}–series, since l ≥ 0. It is directly verifiable that the two resulting expressions sum to zero.

It is easily seen that the conditions on l in (9) allow min(k+1, n−k) possible values for l. Thus, we have min(k + 1, n − k) independent linear combinations of columns which vanish in all coordinates except possibly in the (s + 1)-th coordinate. (Recall that (9) is valid only for i ≠ s + 1). It is clear that an appropriate combination of two of these linear combinations vanishes in every coordinate. So we have min(k, n−k−1) independent linear combinations vanishing at m = −k − \frac{1}{2} for k = 1, …, n − 2, which proves (8).

7. The “integral” factors of det (B_{ij})

In this section we prove the following result (see (5) for the definition of B_{ij}).

\frac{1}{(n + m − s)(s + m)} ∏_{k=0}^{n} (m + k)^{min(k+1,n−k+1)}divides det (B_{ij}) as a polynomial in m.

(10)

We use linear combinations of the rows of B_{ij} that vanish for m = −k. Without loss of generality, we can assume that s ≤ \frac{n}{2} because both the final result (see Theorem 1) and the number of rhombus tilings of the original graph are invariant under the transformation s → n − s.

Most of the factors (m + k) can be taken out directly from the rows of B_{ij}. In fact, it is easily seen that row i is divisible by (m + 1 − i + n)_{2i−n−1} for 2i ≥ n + 2. The product of these terms equals ∏_{k=1}^{n−1} (m + k)^{min(k,n−k)}. The matrix (C_{ij}) which remains after taking out these factors from B_{ij} looks as follows:

C_{ij} = \begin{cases}(n + j + 1 − 2s)_{n−j}(s + m + 1 − j)_{j−1} \\ i = s + 1, \\ (n + 2 + j − 2i)_{n−j}(i + m + 1 − j)_{j−2i+n}(2m + n + 1 − j) \\ i ≠ s + 1, 2i ≥ n + 2, \\ (n + 2 + j − 2i)_{n−j}(i + m + 1 − j)_{j−1}(2m + n + 1 − j) \\ i ≠ s + 1, 2i < n + 2. \end{cases}

To finish the proof of (10) we have to find for each value k = 0, …, n, k ≠ s and k ≠ (n − s) one vanishing linear combination of the rows of C_{ij}.

We start with the case k < s. We claim that

∑_{i=k+1}^{s} ((−1)^{i−k+1}\binom{s − k − 1}{i − k − 1}\frac{(n + \frac{3}{2} − i)_{i−k−1}(n − i + 1)_{i−k−1}}{(s + \frac{1}{2} − i)_{i−k−1}(n − k − i + 1)_{i−k−1}} C_{ij}|_{m=−k})

+ (−1)^{s−k+2} \frac{2(n + \frac{3}{2} − s − 1)_{s−k}(n − s + 1)_{s−k−1}}{(\frac{1}{2})_{s−k−1}(n − k − s + 1)_{s−k−1}} C_{s+1,j}|_{m=−k} = 0. (11)

If −k + s − j < 0 then the terms (i − k + 1 − j)_{j−1} and (s − k + 1 − j)_{j−1} (which are factors of C_{ij}|_{m=−k} and C_{s+1,j}|_{m=−k}, respectively) are zero for all occurring indices.

If −k + s − j ≥ 0, we reverse the order of summation in the sum and write (11) in hypergeometric form,

(−1)^{−k+s} \frac{(n + \frac{3}{2} − s)_{s−k−1} (1 − j − k + s)_{j−1}}{(\frac{1}{2})_{s−k−1} (n − k − s + 1)_{s−k−1}(2 + j + n − 2 s)_{n−j} (n − s + 1)_{s−k−1}}

× (−1 + j + 2k − n)_{3}F_{2}[\begin{matrix}\frac{1}{2}, n − k + 1 − s, j + k − s \\ 1 + \frac{j}{2} + \frac{n}{2} − s, \frac{3}{2} + \frac{j}{2} + \frac{n}{2} − s \end{matrix}; 1]

+ (−1)^{s−k+2} \frac{2(n + \frac{3}{2} − s − 1)_{s−k}(n − s + 1)_{s−k−1}}{(\frac{1}{2})_{s−k−1}(n − k − s + 1)_{s−k−1}} C_{s+1,j}|_{m=−k} = 0.

Now we can apply the Pfaff–Saalschütz summation formula ([9], (2.3.1.3); Appendix (III.2)),

3F_{2}[\begin{matrix}a, b, −n \\ c, 1 + a + b − c − n \end{matrix}; 1] = \frac{(−a + c)_{n} (−b + c)_{n}}{(c)_{n} (−a − b + c)_{n}} ,  (12)

where n is a nonnegative integer.

It is easily verified that the resulting sum of two terms equals zero.

The case k > n − s is quite similar. We claim that

∑_{i=n−k+1}^{s} (−1)^{i−n+k+1}\binom{s − n + k − 1}{i − n + k − 1}\frac{(n + \frac{3}{2} − i)_{i−n+k−1}(n − i + 1)_{i−n+k−1}}{(s + \frac{1}{2} − i)_{i−n+k−1}(k − i + 1)_{i−n+k−1}} C_{ij}|_{m=−k}

− (−1)^{s−n+k+2} \frac{2(n + \frac{3}{2} − s − 1)_{s−n+k}(n − s + 1)_{s−n+k−1}}{(\frac{1}{2})_{s−n+k−1}(k − s + 1)_{s−n+k−1}} C_{s+1,j}|_{m=−k} = 0. (13)

Converting the reversed sum to hypergeometric form gives

(−1)^{s+k−n}(−1 + j + 2k − n)(2 + j + n − 2 s)_{n−j} (1 + n − s)_{−1+k−n+s}

× \frac{(\frac{3}{2} + n − s)_{−1+k−n+s}(1 − j − k + s)_{−1+j}}{(\frac{1}{2})_{−1+k−n+s}(1 + k − s)_{−1+k−n+s}}

×_{3}F_{2}[\begin{matrix}j + k − s, \frac{1}{2}, 1 − k + n − s \\ 1 + \frac{j}{2} + \frac{n}{2} − s, \frac{3}{2} + \frac{j}{2} + \frac{n}{2} − s \end{matrix}; 1] .

Again, the Pfaff–Saalschütz summation formula (12) is applicable because −(1 − k + n − s) is a nonnegative integer. It is easily checked that the resulting terms sum to 0.

So our remaining task is the case s < k < n − s. For s < k ≤ \frac{n}{2} we consider the following linear combination,

\begin{matrix}⌊ & \frac{n+1}{2} & ⌋ \\ ∑ \\ i=k+1 \end{matrix} (−4)^{n−i} \frac{(s − i + 1)_{i−k−1}(s − n + \frac{1}{2})_{n−k−1}}{(2n − 2i + 1)!(s + \frac{1}{2} − i)_{i−k−1}(n + 1 − s)_{−k}} (i − k)_{n+1−2i}C_{ij}|_{m=−k}

+ \begin{matrix}⌊ & \frac{n+1+j}{2} & ⌋ \\ ∑ \\ i=⌊\frac{n+3}{2}⌋ \end{matrix} (−4)^{n−i} \frac{(s − i + 1)_{i−k−1}(s − n + \frac{1}{2})_{n−k−1}}{(2n − 2i + 1)!(s + \frac{1}{2} − i)_{i−k−1}(n + 1 − s)_{−k}} C_{ij}|_{m=−k}

− C_{s+1,j}|_{m=−k} = 0. (14)

Now, both the term (i − k)_{n+1−2i}C_{ij}|_{m=−k} which is part of the first sum in (14) and

the term C_{ij}|_{m=−k} which is part of the second sum in (14) are equal to (n + 2 + j −

2i)_{n−j}(i − k + 1 − j)_{j−2i+n}(−2k + n + 1 − j), so we can combine the two sums into one sum of a hypergeometric term.

We distinguish two cases according to the parity of n − j. In both cases we reverse the sum and convert it to hypergeometric form. The resulting two hypergeometric series are

3F_{2} [\begin{matrix}1 + l + m, −l − m, \frac{1}{2} + l − n + s \\ \frac{3}{2}, 1 + l − n + s \end{matrix}; 1]  for n − j = 2l,
3F_{2} [\begin{matrix}1 + l + m, −1 − l − m, \frac{1}{2} + l − n + s \\ \frac{1}{2}, 1 + l − n + s \end{matrix}; 1]  for n − j = 2l + 1.

So we can use the Pfaff–Saalschütz summation formula (12) again. It is easily verified that in both cases the resulting terms add to zero.

The case \frac{n}{2} < k < n − s is handled similarly. We claim that the following sum equals zero,

\begin{matrix}⌊ & \frac{n+1}{2} & ⌋ \\ ∑ \\ i=n−k+1 \end{matrix} (−4)^{n−i} \frac{(s − i + 1)_{i−n+k−1}(s − n + \frac{1}{2})_{k−1}(i − k)_{n+1−2i}}{(2n − 2i + 1)!(s + \frac{1}{2} − i)_{i−n+k−1}(n + 1 − s)_{−n+k}} C_{ij}|_{m=−k}

+ \begin{matrix}⌊ & \frac{n+j+1}{2} & ⌋ \\ ∑ \\ i=⌊\frac{n+3}{2}⌋ \end{matrix} (−4)^{n−i} \frac{(s − i + 1)_{i−n+k−1}(s − n + \frac{1}{2})_{k−1}}{(2n − 2i + 1)!(s + \frac{1}{2} − i)_{i−n+k−1}(n + 1 − s)_{−n+k}} C_{ij}|_{m=−k}

− (−1)^{n}C_{s+1,j}|_{m=−k} = 0. (15)

Again, we write the two sums as one single sum, distinguish two cases according to the parity of n−j, and reverse the order of summation. Conversion to hypergeometric form of the resulting sums gives

3F_{2} [\begin{matrix}1 + l − k, −l + k, \frac{1}{2} + l − n + s \\ \frac{3}{2}, 1 + l − n + s \end{matrix}; 1]  for n − j = 2l,
3F_{2} [\begin{matrix}1 + l − k, −1 − l + k, \frac{1}{2} + l − n + s \\ \frac{1}{2}, 1 + l − n + s \end{matrix}; 1]  for n − j = 2l + 1.

The Pfaff–Saalschütz summation formula (12) can be applied in both cases. It is easily seen that the results vanish after subtraction of (−1)^{n}C_{s+1,j}|_{m=−k}. Thus (10) is proved.

8. The degree and the leading coefficient

We have to find the degree and the leading coefficient of the determinant det (B_{ij}) as a polynomial in m. The degree of B_{ij} is j − 1 for i = s + 1 and j else. Therefore, the degree of det (B_{ij}) is at most \binom{n+1}{2} − 1, which is easily seen to be the number of linear factors we have found to divide det (B_{ij}). Therefore det (B_{ij}) is the product of the linear factors and the leading coefficient.

To compute the leading coefficient we look at the leading coefficient of each entry. The leading coefficients of the entries give the matrix D, with

D_{ij} = (x_{i} + n + j)_{n−j}, where x_{i} = \begin{cases}2 − 2i & i ≠ s + 1 \\ 1 − 2s & i = s + 1. \end{cases}

This matrix can be transformed by column reduction to (x_{i}^{n−j}), but this is just the Vandermonde determinant ∏_{1≤i<j≤n} (x_{i} − x_{j}). Plugging in the values of the x_{i} gives the following result.

s 2m + 1 {  m { m + 1 {

Figure 7. The hexagon in the case of odd sidelength and the two halves R̃^{+} and R̃^{−}. m = 2, n = 3, s = 2.

The leading coefficient of det (B_{ij}) is

\frac{2^{\binom{n−1}{2}} H(n)(2n − 2s − 1)!!(2s − 1)!!}{(n − s − 1)!s!}.  (16)

(8), (10) and (16) immediately give equation (7). Thus the proof of Lemma 6 is complete.

9. Proof of Theorem 2

If the side divided by the symmetry axis has odd length, the position s of the missing triangles ranges from 1 to n (see Figure 7). We can form the inner dual graph and denote it by G̃. Now we can proceed analogously to Section 3, break the graph in two parts G̃^{+} and G̃^{−} with the help of the matchings factorization theorem (see Lemma 3). We convert G̃^{+} and G̃^{−} back to regions R̃^{+} and R̃^{−} of triangles and have to count rhombus tilings again. R̃^{+} and R̃^{−} are shown in an example in Figure 7.

Thus, we have

M(G̃) = 2^{n−1}M(R̃^{+})M(R̃^{−}).  (17)

Now we reduce the evaluation of M(R̃^{+}) to the evaluation of M(R^{+}), which we have already done in Lemma 5. M(R^{+}) and M(R̃^{+}) are related in the following way. The tiles of the upper half R^{+} of the hexagon with sides n, n, 2m, n, n, 2m (as exemplified in Figure 1) sharing an edge with the border of length m are enforced as shown in

Figure 8. M(R̃^{+}(3, 2)) equals M(R^{+}(2, 2)).

Figure 9. A tiling and lattice paths for R̃^{−}.

Figure 8 (the forced tiles are shaded). After removal of these tiles we are left with the upper half R̃^{+} of the hexagon with sides n − 1, n − 1, 2m + 1, n − 1, n − 1, 2m + 1. Thus, we have M(R^{+}(n, m)) = M(R̃^{+}(n − 1, m)) and Lemma 5 implies directly the following result:

M(R̃^{+}) = \frac{H(n + 1) ∏_{2≤i≤j≤n+1} (2m + 2j − i)}{∏_{j=1}^{n+1} (2j − 2)!}.  (18)

The lower half R̃^{−} can be turned into a determinant in a manner analogous to Section 5 (see Figure 9).

The starting and end points and the resulting determinant equal

R̃_{i} = \begin{cases}(2i − 1, i + m) & for i ≠ s \\ (2s − 2, s + m − 1) & for i = s \end{cases}
S̃_{j} = (n + j − 1, j − 1)

Ã_{ij} = \begin{cases}\frac{1}{2}\binom{n + m − i}{m + i − j + 1} + \binom{n + m − i}{m + i − j} & for i ≠ s \\ \binom{n + m − s + 1}{m + s − j} & for i = s. \end{cases}

Since the original problem and the claimed final result are invariant under n+1−s → s, we can assume s ≠ n. (The case n = s = 1 is trivial to check.) Then

Ã_{nj} = \frac{1}{2}\binom{m}{m + n − j + 1} + \binom{m}{m + n − j}

= \begin{cases}1 for j = n, \\ 0 else, \end{cases}

since m is a nonnegative integer. It is easily seen that

Ã_{ij}(n, m, s) = A_{ij}(n − 1, m + 1, s − 1) for i, j < n,

where A_{ij} is defined in equation (4). We expand det_{1≤i,j≤n}(Ã_{ij}(n, m, s)) with respect to row n and get

det_{1≤i,j≤n}(Ã_{ij}(n, m, s)) = det_{1≤i,j≤n−1}(A_{ij}(n − 1, m + 1, s − 1)),

Hence,

M(R̃^{−}(n, m, s)) = M(R^{−}(n − 1, m + 1, s − 1)).

Thus, Lemma 6 yields

M(R̃^{−}) = \frac{2^{\binom{n−2}{2}−1} H(n − 1)(2n − 2s − 1)!!(2s − 3)!!}{(n − s)!(s − 1)! ∏_{i=0}^{n−2} (2i + 1)!}

× ∏_{k=1}^{n−3} (m + 1 + k + \frac{1}{2})^{min(k,n−2−k)} ∏_{k=0}^{n−1} (m + 1 + k)^{min(k+1,n−k)} (19)

Now we substitute the results of equations (18) and (19) in (17). We get

M(G̃) = 2^{n−1}M(R̃^{+})M(R̃^{−})  (20)

= \frac{2^{\binom{n−1}{2}−1} H(n − 1) H(n + 1)(2n − 2s − 1)!!(2s − 3)!!}{(n − s)!(s − 1)! ∏_{j=0}^{n} (2j)! ∏_{i=0}^{n−2} (2i + 1)!}

× ∏_{k=2}^{n−2} (m + k + \frac{1}{2})^{min(k−1,n−k−1)} ∏_{k=1}^{n} (m + k)^{min(k,n−k+1)} ∏_{2≤i≤j≤n+1} (2m + 2j − i),

which can easily be transformed to the expression in Theorem 2.

References

[1] Mihai Ciucu, Enumeration of perfect matchings in graphs with reflective symmetry, J. Combin. Theory Ser. A 77 (1997), 67–97.

[2] G. David and C. Tomei, The problem of the calissons, Amer. Math. Monthly. 96 (1989), 429– 431.

[3] M. Fulmek and C. Krattenthaler The number of rhombus tilings of a symmetric hexagon which contain a fixed rhombus on the symmetry axis, II, preprint.

[4] I.M. Gessel and X. Viennot, Determinant, paths and plane partitions, preprint, (1989).

[5] C. Krattenthaler, An alternative evaluation of the Andrews–Burge determinant, in: Mathematical Essays in Honor of Gian-Carlo Rota, B. E. Sagan, R. P. Stanley, Progress in Math., vol. 161, Birkhäuser

[6] C. Krattenthaler, Generating functions for plane partitions of a given shape, Manuscripta Math. 69, (1990), 173–202.

[7] P.A. MacMahon, Combinatory Analysis, vol. 2, Cambridge University Press, (1916); reprinted by Chelsea, New York, (1960).

[8] J. Propp, Twenty open problems on enumeration of matchings, manuscript, (1996).

[9] L. J. Slater, Generalized hypergeometric functions, Cambridge University Press, Cambridge, (1966).

[10] J. R. Stembridge, Nonintersecting paths, pfaffians and plane partitions, Adv. in Math. 83 (1990) 96—131.