Bisbee-Baldwin Corporation v. Tomlinson

• Decision Date: 25 June 1963
• Docket Number: No. 19567.,19567.
• Citation: 320 F.2d 929
• Parties: BISBEE-BALDWIN CORPORATION, Appellant, v. Laurie E. TOMLINSON, District Director of Internal Revenue for the District of Florida, Appellee.
• Court: U.S. Court of Appeals — Fifth Circuit

320 F.2d 929 (1963)

BISBEE-BALDWIN CORPORATION, Appellant, v. Laurie E. TOMLINSON, District Director of Internal Revenue for the District of Florida, Appellee.

No. 19567.

United States Court of Appeals Fifth Circuit.

June 25, 1963.

Chester Bedell, C. Harris Dittmar, Jacksonville, Fla., Bedell, Bedell & Dittmar, Jacksonville, Fla., of counsel, for appellant.

David O. Walter, Meyer Rothwacks, Attys., Dept. of Justice, Louis F. Oberdorfer, Asst. Atty. Gen., Lee A. Jackson, Atty., Dept. of Justice, Washington, D. C., Edward F. Boardman, U. S. Atty., Miami, Fla., William J. Hamilton, Jr., Asst. U. S. Atty., Jacksonville, Fla., for appellee.

Before WISDOM and BELL, Circuit Judges, and JOHNSON, District Judge.

WISDOM, Circuit Judge.

The question for decision is whether payments the taxpayer received for the termination of mortgage servicing contracts are taxable as ordinary income or as capital gains.¹

This Court has recently decided three other cases concerning the tax effects of the termination of similar contracts: Nelson Weaver Realty Co. v. Commissioner, 5 Cir.1962, 307 F.2d 897; United States v. Eidson, 5 Cir.1962, 310 F.2d 111; Commissioner v. Maurice L. Killian, 5 Cir.1963, 314 F.2d 852. In Weaver, Judge Cameron, for the majority of the Court, held: "It cannot be doubted that the sum total of the ingredients of the mortgage company's longstanding relationship with * * * its clientele constitutes a property right which is the equivalent of goodwill * * * a capital asset * * * entitled to capital gain treatment" on its sale. Judge Rives dissented: The "part allocable to the right to receive service fees should be taxable as an ordinary income."* In Eidson Judge Tuttle, for the Court, held that it "makes no difference whether the assignment of rights under the contract * * * be denominated a sale"; the taxpayers assigned the right to receive "net profit of operations during the remainder of the life of the contract", which must be treated as ordinary income. Killian followed Weaver "to the extent that the facts * * * established a sale of good will", but the property sold consisted of "expirations" not a bundle of rights as in Weaver, Eidson, and the case now before the Court.²

* Footnote 6 of the majority opinion in Weaver points out that the question of allocation according to fragments was not presented or argued before the Tax Court or this Court.

A majority of this Court conclude that they cannot bridge the gulf between Weaver and Eidson. Since we must choose between the two, we choose to follow Eidson on principle and the solution to the problem suggested by the dissent in Weaver.

I.

The facts are not in dispute.

Bisbee-Baldwin, the taxpayer, is in the mortgage banking business. Most of its loans are secured by mortgages on residential property in Jacksonville, Florida. After making a loan, the company invariably assigns the mortgage to an institutional investor. The essential profit-making element is the investor's agreement to employ the mortgage company as its agent to service the mortgages. The company receives no profit on the assignment of a mortgage but earns an annual commission of one-half of one per cent of the principal outstanding balance of the mortgages serviced. The servicing activities generate other business. For example, the company often writes fire insurance on the property mortgaged, acts as real estate broker when the property is sold, and serves as property manager when a mortgage is foreclosed. Escrow deposits by the mortgagors enhance its credit standing, a substantial benefit since the company must borrow large sums from the banks in the operation of its affairs. Thus the success of the mortgage servicing business depends upon the amount of mortgage indebtedness it services.

The taxpayer had no right to assign the servicing agreement and could not demand any payment from a successor servicing agent if the investor transferred its business to another company. The taxpayer was not the exclusive agent for any investor, even in the Jacksonville area serviced by it. Each investor had the right to enter into similar agreements with other servicing agents, and the taxpayer had the right to assign and service mortgages for other investors.

During the fiscal year ending April 30, 1957, various investors cancelled servicing agreements with the taxpayer, and gave the business to other agents. When an investor cancels such an agreement without cause, it is customary for the investor to pay a termination fee equal to one per cent of the principal balance of the mortgages then being serviced by the mortgage company. In this case, several of the taxpayer's agreements with investors expressly provided for such a termination fee. The taxpayer received net termination fees of $206,454.63. The investors paid this sum to Bisbee-Baldwin, but were reimbursed by the new servicing agents for the amount of the termination fees paid to the taxpayer. In substance, therefore, the mortgage servicing was transferred from Bisbee-Baldwin to other agents for, as the district court found, the cancellations would not have taken place had the successor mortgage servicing agents not agreed to reimburse the investors in the amount of the termination fees.

The district court found that:

"Plaintiff kept extensive files containing information concerning the mortgages which it serviced. When an investor canceled an agreement the files containing information concerning the mortgages assigned to that investor were turned over either to the investor or to the new servicing agent designated by the investor. Those files were of value to the Plaintiff in obtaining the indirect advantages of servicing mortgages."

Bisbee-Baldwin reported the termination fees as a net long-term capital gain of $206,454.43. The Commissioner ruled that the sum was ordinary income and assessed a deficiency of $45,705.35 against Bisbee-Baldwin. The taxpayer paid the deficiency and then brought suit for refund. The district court, citing the "Weaver case as a situation very similar * * * agreed with the conclusion of the Tax Court that a mortgaging servicing contract is not a capital asset". The district court also held that the "termination of the mortgage servicing contracts by the various investors did not constitute sales or exchanges by the plaintiff." Bisbee-Baldwin appeals from the judgment of the district court.

II.

The fact that contractual rights in a mortgage servicing agency constitute a species of "property" under state law affords no assistance in determining whether such rights are capital assets.³ As the Supreme Court stated in Commissioner v. Gillette Motor Transport Co., 1960, 364 U.S. 130, 134-135, 80 S.Ct. 1497, 1500, 1501, 4 L.Ed.2d 1617:

"While a capital asset is defined in § 117(a) (1) as `property held by the taxpayer,\' it is evident that not everything which can be called property in the ordinary sense and which is outside the statutory exclusions qualifies as a capital asset. This Court has long held that the term `capital asset\' is to be construed narrowly in accordance with the purpose of Congress to afford capital-gains treatment only in situations typically involving the realization of appreciation in value over a substantial period of time, and thus to ameliorate the hardship of taxation of the entire gain in one year. * * * Thus the Court has held that an unexpired lease, * * * corn futures, * * * and oil payment rights, * * * are not capital assets even though they are concededly `property\' interests in the ordinary sense."

In Hort v. Commissioner, 1941, 313 U.S. 28, 61 S.Ct. 757, 85 L.Ed. 1168, the Court held that amounts received in cancellation of a lease were taxable as ordinary income to a lessor. The Court said:

"Simply because the lease was `property\' the amount received for its cancellation was not a return of capital, quite apart from the fact that `property\' and `capital\' are not necessarily synonymous in the Revenue Act of 1932 or in common usage. Where, as in this case, the disputed amount was essentially a substitute for rental payments which § 22(a) expressly characterizes as gross income, it must be regarded as ordinary income, and it is immaterial that for some purposes the contract creating the right to such payments may be treated as `property\' or `capital.\'" (313 U.S. 31, 61 S.Ct. 758)

This statement agrees with comparable statements in other cases that the capital gains section is an exception from the usual requirements and must be narrowly construed. See Corn Products Co. v. Commissioner, 1955, 350 U.S. 46, 76 S.Ct. 20, 100 L.Ed. 29; C. I. R. v. P. G. Lake, 1958, 356 U.S. 260, 78 S.Ct. 691, 2 L.Ed.2d 743; Burnet v. Harmel, 1932, 287 U.S. 103, 53 S.Ct. 74, 77 L.Ed. 199.

The question is, what do the mortgage servicing rights under the contracts represent. If they represent the right to earn future income in the form of commissions for services rendered, then the sum received for the cancellation of the contracts and the transfer of rights is ordinary income.

In Roscoe v. Commissioner, 5 Cir.1954, 215 F.2d 478, a sum received by taxpayers for their real-estate corporation stock in excess of the amount received by other shareholders was held to be ordinary income representing the commission which the taxpayers would have been entitled to had the land represented by the stock been sold directly. The Tenth Circuit recently reached a similar result in Wiseman v. Halliburton Oil Well Cementing Co., 10 Cir.1962, 301 F.2d 654. There the taxpayer had agreed to the termination of an exclusive license to use and to grant sub-licenses for a patented process in exchange for a nonexclusive license and one-third of the royalties received from third party licenses. The Court held that in substance the taxpayer had received the present value of income which it would otherwise have received in the future through the sublicenses. In so holding the Tenth Circuit expressly...