Calloway v. Comm'r

• Decision Date: 23 August 2012
• Docket Number: No. 11–10395.,11–10395.
• Citation: 691 F.3d 1315,110 A.F.T.R.2d 2012,23 Fla. L. Weekly Fed. C 1462
• Parties: Lizzie W. CALLOWAY, Albert Calloway, Petitioners–Appellants, v. COMMISSIONER OF IRS, Respondent–Appellee.
• Court: U.S. Court of Appeals — Eleventh Circuit

691 F.3d 1315

110 A.F.T.R.2d 2012-5661

23 Fla. L. Weekly Fed. C 1462

Lizzie W. CALLOWAY, Albert Calloway, Petitioners–Appellants,

v.COMMISSIONER OF IRS, Respondent–Appellee.

No. 11–10395.

United States Court of Appeals,Eleventh Circuit.

Aug. 23, 2012.

Brian G. Isaacson, Isaacson Wilson, Seattle, WA, for Petitioners–Appellants.

Andrew Weiner, Kenneth L. Greene, Gilbert Steven Rothenberg, Tax Div., App. Sec., U.S. Dept. of Justice, William J. Wilkins, Chief Counsel–IRS, Robert R Di Trolio, Washington, DC, for Respondent–Appellee.

Petition for Review of a Decision of the U.S. Tax Court.

Before CARNES, PRYOR and RIPPLE,^* Circuit Judges.

RIPPLE, Circuit Judge:

Albert and Lizzie Calloway seek review of a judgment of the Tax Court sustaining the Commissioner's determination of a deficiency, an accuracy-related penalty and a penalty for filing a delinquent tax return. For the following reasons, we affirm the judgment of the Tax Court.

I

A. Facts

Albert Calloway worked for IBM for a number of years and acquired IBM stock by exercising his employee stock options. During 2001, Bert Falls, the Calloways' financial adviser, introduced Mr. Calloway to a program operated by Derivium Capital, LLC (“Derivium”). Under that program, Derivium would “lend” a client ninety percent of the value of securities that the client pledged to Derivium as collateral. During the term of the non-recourse loan, Derivium had no restrictions on its use of the collateral. At the end of the loan's term, the client had three options: (1) He could reclaim the collateral by paying the principal and accrued interest; (2) He could surrender the collateral to Derivium; or (3) He could refinance. Mr. Calloway testified that the loan program was attractive to him because, had he sold his stock, he would have had to pay twenty percent in capital gains tax; under the Derivium program, however, he received ninety percent of the stock's fair market value.

Before entering Derivium's program, Mr. Calloway consulted Falls as well as a tax advisor, who provided Mr. Calloway with a copy of a memorandum from Robert J. Nagy to Charles D. Cathcart, president of Derivium. In the memo, Nagy, who identified himself as a certified public accountant, opined that, although there were no absolute guarantees, there was a “solid basis for the position that these transactions are, in fact, loans.”¹ Mr. Calloway later testified that “[i]t gave [him] comfort to know that a decision had been considered and that this was in fact a loan and not anything other, and it's a nonrecourse loan.”² Mr. Calloway admitted, however, that he did not do any research to find out who Nagy was, that he did not know whether Nagy was associated with Derivium and that he did not “have any idea what [Nagy's] training or background [wa]s.”³

The details of Mr. Calloway's arrangement with Derivium are set forth in three documents: “Master Agreement to Provide Financing and Custodial Services” (“Master Agreement”), “Schedule D Disclosure Acknowledgment and Broker/Bank Indemnification” (“Schedule D”) and “Schedule A–1 Proper Description and Loan Terms” (“Schedule A–1”).

The Master Agreement provides:

This Agreement is made for the purpose of engaging [Derivium] to provide or arrange financing(s) and to provide custodial services to the Client, with respect to certain properties and assets (“Properties”) to be pledged as security, the details of which financing and Properties are to be set out in loan term sheets and attached hereto as Schedule(s) A....[⁴]

Paragraph 3 of the Master Agreement relates to the “Funding Of [the] Loan” and provides in relevant part:

The Client understands that by transferring securities as collateral to [Derivium] and under the terms of the Agreement, the Client gives [Derivium] and/or its assigns the right, without requirement of notice to or consent of the Client, to assign, transfer, pledge, repledge, hypothecate, rehypothecate, lend, encumber, short sell, and/or sell outright some or all of the securities during the period covered by the loan. The Client understands that [Derivium] and/or its assigns have the right to receive and retain the benefits from any such transactions and that the Client is not entitled to these benefits during the term of a loan....[⁵]

The Master Agreement also includes a provision that allowed either party to terminate the agreement “at any time prior to the funding of a loan.”⁶

Schedule A–1 details the terms of the loan. Specifically, the loan amount was ninety percent of the fair market value at the time of closing; the estimated value of the collateral at that time was $105,444.90. The interest rate to be charged was ten-and-one-half percent, compounded annually; the interest accrued until, and was due at, maturity. Any dividends on the pledged collateral were to “be received as cash payments against interest due.” ⁷ The loan could not be pre-paid, and the lender could not seek recourse against the borrower, only the collateral. The closing date was “[u]pon receipt of securities and establishment of [Derivium's] hedging transactions.”[⁸]

Mr. Calloway executed the Master Agreement and attached schedules on August 8, 2001, and authorized the transfer of 990 shares of his IBM stock to Derivium's account with Morgan Keegan & Company, Inc., on the following day. Cathcart, as president of Derivium, signed the Master Agreement and schedules on August 10, 2001.

On August 17, 2001, Derivium's operations office sent Mr. Calloway two documents. The first was a valuation confirmation indicating that Derivium had received the stock, valued at $104,692.50, into its account. The second document, titled “Activity Confirmation,” indicated that, as of August 17, 2001, Derivium had hedged the IBM stock for slightly less, $103,984.70, yielding an “Actual Loan Amount” of $93,586.23. On August 21, 2001, Derivium sent to Mr. Calloway a letter informing him that the proceeds of the loan were sent to him according to the wire transfer instructions he had provided a few days earlier. On the same date, $93,586.23 was credited to Mr. Calloway's credit union account. Previously, on August 17, Derivium had exercised its right to sell the stock without giving notice to Mr. Calloway.⁹

During the period of time covered by the loan, Mr. Calloway received quarterly and year-end account statements. Each quarterly statement set forth the loan balance at the beginning of the quarter, indicated the interest accrued during the quarter and credited the account for the dividends paid during the quarter to yield the end-of-quarter loan balance. The statement also provided the end-of-quarter collateral value. Mr. Calloway did not receive any tax statements reflecting dividend income (Form 1099–DIV), nor did he report on his tax returns any dividend income earned from the 990 shares of IBM stock.

In a letter dated July 8, 2004, Derivium informed Mr. Calloway that the loan would mature on August 21, 2004. Consistent with the Master Agreement and accompanying schedules, the letter stated that Mr. Calloway could either (1) pay the maturity amount of $124,429.09 and recover his collateral, (2) renew or refinance the transaction for an additional term or (3) surrender the collateral. On July 27, 2004, Mr. Calloway returned the response form to Derivium indicating that he was “surrender[ing his] collateral in satisfaction of [his] entire debt obligation.”¹⁰

On February 11, 2004, Mr. Calloway and his wife filed their joint federal income tax return for the year 2001—nearly two years past the original filing deadline. On the return, they did not report the money received from Derivium in exchange for the stock. From the time of the loan until the surrender of the collateral in August 2004, the Calloways did not declare on their tax returns any dividend income generated by the 990 shares of IBM stock. The Calloways also did not report on their 2004 return the surrender of their collateral to Derivium, either by declaring capital gains from the sale of securities or by declaring a forgiveness of indebtedness.

The Internal Revenue Service (“the IRS” or “the Service”) issued a notice of deficiency to the Calloways for failing to include the income from the sale of the IBM stock on their 2001 income tax return. It also assessed two penalties for failure to timely file a return and for significant understatement of income.

The Calloways filed a petition for a redetermination of income tax deficiency and penalties with the Tax Court.

B. Proceedings in the Tax Court

1.

The Calloways framed the issues before the court in their pretrial memorandum. They argued that the mere fact that Mr. Calloway entered into the loan agreement with Derivium to avoid paying capital gains tax was not license for the Commissioner to recast the arrangement as a sale.¹¹ “In this case,” the Calloways maintained, they “chose to enter into a bona fide and actual loan agreement with Derivium.”¹²

The Calloways also argued that “the Commissioner [could] not rely upon a series of events which were ... outside of [the] Calloway[s'] control to recast the loan as a sale.”¹³ According to the Calloways, the Commissioner's efforts to characterize the 2001 transaction as a sale were based on Derivium's actions. The Calloways noted that they “did not discover that the shares of stock were inappropriately sold by Der[i]vium until after 2004.”¹⁴ It therefore “would be unfair,” they continued, “to expect [them] to include the proceeds of the stock sale in their income taxes for the years that they did not, in good faith, realize such income.”¹⁵

Finally, they maintained that “[a]ny plain reading of the Derivium loan contract demonstrates that the contract is clearly a loan agreement entered into by two independent, arms-length, unrelated parties, rather than an agreement to sell the stock pledged under the loan agreement.”¹⁶ They claimed that this position was “further fortified by the existence of a long-standing Internal Revenue...