QBUS1040: Foundations of Business Analytics MSE Solution Semester 2, 2020 Tutorial time: Tutor’s name: Your SID: (For QBUS1040 sta↵ only) Question: 1 2 3 4 5 6 7 Total Points: 10 10 10 10 10 10 10 70 Score: i QBUS1040 MSE Solution Semester 2, 2020 1. (a) (8 points) Use the Cauchy-Schwarz inequality to prove that rms(x)  avg(x)  rms(x) for all n-vectors x. (b) (1 point) What are the conditions on x to have equality in the upper bound? (c) (1 point) When do we have equality in the lower bound? Solution: (a) Apply Cauchy-Schwarz inequality to avg(x) |1Tx| n  k1kkxk n (1)  p nkxk n , since k1k = p 12 + 12 + · · · + 12 = pn (2)  p nkxkp n p n (3) |1Tx| n  kxkp n (4) This is equivalent to writing rms(x)  avg(x)  rms(x) (5) (b) In other words we wish to know what are the conditions on x for avg(x) = rms(x) to hold. We know that rms(x)2 = avg(x)2 + std(x)2. Hence, rms(x) = avg(x) when the std(x) = 0. This is only possible when the vector x is a vector of constants. Since the rms(x) will always be positive we must also have that avg(x) is positive. Hence, the two conditions are that x is a vector of positive constants. (c) In other words we wish to show what are the conditions of x for rms(x) = avg(x) to hold. From part (a) we know that std(x) = 0 is one condition required for rms(x) = avg(x) to hold, so x must be a vector of constants. Since the rms(x) is always negative we must have that avg(x) is negative. Hence, the two conditions are that x is a vector of negative constants. Page 1 of 8. QBUS1040 MSE Solution Semester 2, 2020 2. (10 points) Find a matrix A for which Ax = (x2, . . . , xn3, xn2), where x is an n-vector. (Be sure to specify the size of A, and describe all its entries.) Solution: The matrix A is: A = 2666664 0 1 0 . . . 0 0 0 0 0 0 0 1 . . . 0 0 0 0 0 ... ... ... ... ... ... ... ... 0 0 0 0 . . . 1 0 0 0 0 0 0 0 . . . 0 1 0 0 3777775 = 2666664 eT2 eT3 ... eTn3 eTn2 3777775 Here A is a wide (n3)⇥n selector matrix. Equivalently A may be expressed as a matrix where each row is the a unit vector. Marking Scheme • 10 marks - Correct. • 6 marks - Correct Matrix and incorrect dimensions. • 6 marks - Correct dimensions and incorrect matrix. • 2 marks - Reasonable attempt. • 0 mark - Incorrect. 3. (10 points) Matrix B below is an adjacency matrix of a directed graph on eight nodes (we assume the nodes are labeled 1, 2, . . . , 8). B = 266666666664 0 1 1 0 0 1 0 0 1 0 1 0 1 0 0 0 1 1 0 1 0 0 1 0 0 0 1 0 1 0 1 0 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 0 0 0 1 1 0 1 0 0 0 0 0 0 1 0 0 0 377777777775 Find the number of paths of length 4 between nodes 3 and 6. Solution: B46,3 +B 4 3,6 gives the number of paths of length 4 between node 3 and 6, i.e. 28. Page 2 of 8. QBUS1040 MSE Solution Semester 2, 2020 4. An n⇥ n matrix A is called skew-symmetric if AT = A, i.e., its transpose is its negative. (a) (3 points) Find all 2⇥ 2 skew-symmetric matrices. Solution: Let A =  A11 A12 A21 A22 . Since A is skew-symmetric, i.e. AT = A, we have  A11 A21 A12 A22 = A11 A12 A21 A22 ) 8>>><>>>: A11 = A11 A21 = A12 A12 = A21 A22 = A22 After solving the system of equations, we have A11 = A22 = 0, A21 = A12. Thus, A =  0 A12 A12 0 where A12 can be any real number. • 3 marks: Correct; • 1 mark: Reasonable attempt; • 0 mark: No attempt (b) (3 points) Explain why the diagonal entries of a skew-symmetric matrix must be zero. Solution: Let A = 26664 A11 A12 · · · A1n A21 A22 · · · A2n ... ... . . . ... An1 · · · · · · Ann 37775. Since A is skew-symmetric, i.e. AT = A, we have Aii = Aii for all i = 1, . . . , n. Thus, Aii = 0 for all i = 1, . . . , n. In other words, all diagonal entries in a skew-symmetric matrix must be zero. Marking Scheme: • 3 marks: Correct; • 1 mark: Reasonable attempt; • 0 mark: No attempt (c) (4 points) Show that for a skew-symmetric matrix A, and any n-vector x, (Ax) ? x. Hint. First show that for any n⇥ n matrix A and n-vector x, xT (Ax) =Pni=1Pnj=1Aijxixj . Solution: Let A = 26664 A11 A12 · · · A1n A21 A22 · · · A2n ... ... . . . ... An1 · · · · · · Ann 37775 and x = 26664 x1 x2 ... xn 37775. Then, Ax = 26664 Pn j=1A1jxjPn j=1A2jxj ...Pn j=1Anjxj 37775. Thus, xT (Ax) = Pn i=1 Pn j=1Aijxixj . Given A being a skew-symmetric matrix. For i = j, we have Aiix2i = 0 as Aii = 0 for all i = 1, . . . , n. For i 6= j, Aji = Aij implies Aijxixj + Ajixjxi = 0 for i, j = 1, . . . , n. Therefore, xT (Ax) = Pn i=1 Pn j=1Aijxixj = 0. In other words, (Ax) ? x. Marking Scheme: Page 3 of 8. QBUS1040 MSE Solution Semester 2, 2020 • 4 marks: Correct; • 2 marks: Reasonable attempt; • 0 mark: No attempt 5. Let G 2 Rm⇥n represent a contingency matrix of m students who are members of n groups: Gij = ( 1 if student i is in group j, 0 if student i is not in group j. (A student can be in any number of the groups.) (a) (2 points) What is the meaning of the 3rd column of G? (b) (2 points) What is the meaning of the 15th row of G? (c) (2 points) Give a simple formula (using matrices, vectors, etc.) for the n-vector m, where mi is the total membership (i.e., number of students) in group i. (d) (2 points) Interpret (GGT )ij in simple English. (e) (2 points) Interpret (GTG)ij in simple English. Solution: (a) The 3rd group (b) The 15th student (c) GT 1 (1TG is also acceptable, but is a row vector). (d) The total number of groups that have both students i and j. (e) The total number of students in both group i and j. Marking Guideline For each sub-question. • 2 marks: Correct; • 0 mark: Wrong/no attempt Page 4 of 8. QBUS1040 MSE Solution Semester 2, 2020 6. Suppose we run the k-means algorithm on the N n-vectors x1, . . . , xN , to obtain the group repre- sentatives z1, . . . , zk. Define the matrices X = ⇥ x1 · · · xN ⇤ , Z = ⇥ z1 · · · zk ⇤ . X has size n⇥N and Z has size n⇥ k. We encode the assignment of vectors to groups by the k⇥N clustering matrix C, with Cij = 1 if xj is assigned to group i, and Cij = 0 otherwise. Each column of C is a unit vector; its transpose is a selector matrix. (a) (5 points) Give an interpretation of the columns of the matrixXZC, and the squared (matrix) norm kX ZCk2. (b) (5 points) Justify the following statement: The goal of the k-means algorithm is to find an n⇥k matrix Z, and a k⇥N matrix C, which is the transpose of a selector matrix, so that kXZCk is small, i.e., X ⇡ ZC. Solution: (a) With the characteristic of C, the j-th column of X ZC is xj Cg(j)jzg(j) = xj zg(j), where xj is assigned to group g(j). Also, kX ZCk2 = NX j=1 xj zg(j)2 , i.e. the total sum of squared distance of each xj to the centroid of its nearest group zg(j). Marking Scheme: • 2.5 marks for the interpretation of the columns of X ZC. • 2.5 marks for the interpretation of kX ZCk2. • 2.5 marks for incomplete arguments. • 1 mark for attempt. • 0 mark for no attempt. (b) It is true that the goal of the k-means algorithm is to find an n ⇥ k matrix Z, and a k ⇥ N matrix C so that kX ZCk is small. The k-means algorithm aims to assign each vector xj to its nearest group g(j). That is, xj zg(j)2 = mini=1,...,k kxj zik2. Considering all vectors xj , we have NX j=1 xj zg(j)2 = NX j=1 ✓ min i=1,...,k kxj zik2 ◆ = min kX ZCk2 . This is equivalent to minimize Jclust = 1N PN j=1 xj zg(j)2. Marking Scheme: • 2.5 marks for guessing true. • 2.5 marks for correct justification. • 1 mark for attempt. • 0 mark for no attempt. Page 5 of 8. QBUS1040 MSE Solution Semester 2, 2020 7. We wish to compute the product E = ABCD, where A is m ⇥ n, B is n ⇥ p, C is p ⇥ q, and D is q ⇥ r. (a) (5 points) Find all methods for computing E using three matrix-matrix multiplications. For example, you can compute AB, CD, and then the product (AB)(CD). Give the total number of flops required for each of these methods. Hint. There are four other methods. (b) (5 points) Which method requires the fewest flops, with dimensions m = 10, n = 1000, p = 10, q = 1000, r = 100? Solution: (a) - Exact Recall: The complexity of Z = XY with X of size m⇥ p and Y of size p⇥ n is mn(2p 1). With A 2 Rm⇥n, B 2 Rn⇥p, C 2 Rp⇥q and D 2 Rq⇥r, there are five methods in computing the product E = ABCD: • ABCD = (AB)(CD), i.e. compute AB and CD first, then the product (AB)(CD). The complexity of AB + the complexity of CD + the complexity of (AB)(CD) = mp(2n 1) + pr(2q 1) +mr(2p 1); • ABCD = ((AB)C)D, i.e. compute AB first, then the product of (AB)C, finally the product ((AB)C)D. The complexity of AB + the complexity of (AB)C + the complexity of ((AB)C)D = mp(2n 1) +mq(2p 1) +mr(2q 1); • ABCD = (A(BC))D, i.e. compute BC first, then the product of A(BC), finally the product (A(BC))D. The complexity of BC + the complexity of A(BC) + the complexity of (A(BC))D = nq(2p 1) +mq(2n 1) +mr(2q 1); • ABCD = A((BC)D), i.e. compute BC first, then the product of (BC)D, finally the product A((BC)D). The complexity of BC + the complexity of (BC)D + the complexity of A((BC)D) = nq(2p 1) + nr(2q 1) +mr(2n 1); • ABCD = A(B(CD)), i.e. compute CD first, then the product of B(CD), finally the product A(B(CD)). The complexity of CD + the complexity of B(CD) + the complexity of A(B(CD)) = pr(2q 1) + nr(2p 1) +mr(2n 1). Marking Scheme: • 1 mark for each method: 0.5 marks for the arguments and 0.5 for the correct complexity. • 1 mark for attempt. • 0 mark for no attempt. (b) - Exact When m = 10, n = 1000, p = 10, q = 1000, r = 100, the complexity of • ABCD = (AB)(CD) is 2,217,900; • ABCD = ((AB)C)D is 2,388,900; • ABCD = (A(BC))D is 40,989,000; • ABCD = A((BC)D) is 220,899,000; • ABCD = A(B(CD)) is 5,898,000. Page 6 of 8. QBUS1040 MSE Solution Semester 2, 2020 After comparison, ABCD = (AB)(CD) requires the fewest flops. Marking Scheme: • 2.5 marks for guessing ABCD = (AB)(CD) requires the fewest flops. • 0.5 marks for correct complexity of a method (theoretically or numerically). • 1 mark for attempt. • 0 mark for no attempt. (a) - Approximate Recall: The complexity of Z = XY with X of size m⇥ p and Y of size p⇥ n is approximately 2mnp. With A 2 Rm⇥n, B 2 Rn⇥p, C 2 Rp⇥q and D 2 Rq⇥r, there are five methods in computing the product E = ABCD: • ABCD = (AB)(CD), i.e. compute AB and CD first, then the product (AB)(CD). The complexity of AB + the complexity of CD + the complexity of (AB)(CD) = 2mnp + 2pqr + 2mpr; • ABCD = ((AB)C)D, i.e. compute AB first, then the product of (AB)C, finally the product ((AB)C)D. The complexity of AB + the complexity of (AB)C + the complexity of ((AB)C)D = 2mnp+ 2mpq + 2mqr; • ABCD = (A(BC))D, i.e. compute BC first, then the product of A(BC), finally the product (A(BC))D. The complexity of BC + the complexity of A(BC) + the complexity of (A(BC))D = 2npq + 2mnq + 2mqr; • ABCD = A((BC)D), i.e. compute BC first, then the product of (BC)D, finally the product A((BC)D). The complexity of BC + the complexity of (BC)D + the complexity of A((BC)D) = 2npq + 2nqr + 2mnr; • ABCD = A(B(CD)), i.e. compute CD first, then the product of B(CD), finally the product A(B(CD)). The complexity of CD + the complexity of B(CD) + the complexity of A(B(CD)) = 2pqr + 2npr + 2mnr. Marking Scheme: • 1 mark for each method: 0.5 marks for the arguments and 0.5 for the correct complexity. • 1 mark for attempt. • 0 mark for no attempt. (b) - Approximate When m = 10, n = 1000, p = 10, q = 1000, r = 100, the complexity of • ABCD = (AB)(CD) is 2,220,000; • ABCD = ((AB)C)D is 2,400,000; • ABCD = (A(BC))D is 42,000,000; • ABCD = A((BC)D) is 222,000,000; • ABCD = A(B(CD)) is 6,000,000. Page 7 of 8. QBUS1040 MSE Solution Semester 2, 2020 After comparison, ABCD = (AB)(CD) requires the fewest flops. Marking Scheme: • 2.5 marks for guessing ABCD = (AB)(CD) requires the fewest flops. • 0.5 marks for correct complexity of a method (theoretically or numerically). • 1 mark for attempt. • 0 mark for no attempt. Page 8 of 8. 8\LZ[PVU .P]LUTH[YP_ [OLPUK\JLKTH[YP_VULUVYTPZKLÄULKHZMVSSV^Z! >YP[LHM\UJ[PVU[OH[JVTW\[L[OL ^P[OV\[\ZPUN[OL QRUP M\UJ[PVUZVM[OL5\TW`WHJRHNL5HTL`V\Y M\UJ[PVU VWXdeQW_QRUP HUK`V\YM\UJ[PVUZOV\SK[HRLTH[YP_ A HZPUW\[HUKYL[\YU HZV\[W\[ /PU[!@V\TH`ULLK[V\ZLMVYSVVWHUK[OLPUI\PS[ abV() M\UJ[PVU IQ¦[¦]: imSoUW nXmS\ aV nS A = QS.aUUa\([[ 2, 9, 6, -3, 4, -7, -6, -7, 6, 1], [ -6, -8, -9, 0, -4, 5, -5, 8, -4, 5], [ 6, 1, 2, 1, -4, -9, 7, 1, 6, -3], [ 6, -7, -3, -8, 9, -2, 1, 9, -6, -5], [ -8, 2, 8, -5, 9, -8, 1, 0, 2, 9], [ 6, -10, -8, -3, 4, -8, -4, 5, 6, 5], [ -9, 3, -7, 7, -1, 7, -5, -5, 8, 2], [ -3, -3, -6, -5, -1, 9, -10, 1, -10, 5], [ -2, 2, 6, 7, 9, 1, -3, 2, -8, -7], [ 3, 0, -2, 4, 8, 4, -5, 4, -1, -9]]) #AQVZeU def VWXdeQW_QRUP(A): URZ, cRO = A.VhaSe cROXPQ_VXP = [] foU j in UaQge(cRO): cROXPQ_VXP.aSSeQd(VXP(abV(A[i,j]) foU i in UaQge(URZ))) VROXWiRQ_A = Pa[(cROXPQ_VXP) UeWXUn IQ¦[¦]: # AVVeUWiRQ cell (Q8) - SleaVe dR QRW deleWe RU mRYe WhiV cell IQ¦[¦]: # AVVeUWiRQ cell (Q8) - SleaVe dR QRW deleWe RU mRYe WhiV cell IQ¦[¦]: # AVVeUWiRQ cell (Q8) - SleaVe dR QRW deleWe RU mRYe WhiV cell IQ¦[¦]: # AdhRc aVVeUWiRQ cell - SleaVe dR QRW deleWe RU mRYe WhiV cell IQ¦[¦]: # AdhRc aVVeUWiRQ cell - SleaVe dR QRW deleWe RU mRYe WhiV cell IQ¦[¦]: # AdhRc aVVeUWiRQ cell - SleaVe dR QRW deleWe RU mRYe WhiV cell IQ¦[¦]: # AdhRc aVVeUWiRQ cell - SleaVe dR QRW deleWe RU mRYe WhiV cell