A relative asked for advice on how to publish a math book he had written. I've included my answers below in the hope it might help other aspiring writers.
I would strongly suggest traditional publishing for a math book. You are correct that traditional publisher have access to the proper sales channels. In fact, academia seldom buys self-published books, so traditional publishing is your best, and possibly only, option.
Many people say you must have an agent to traditionally publish. This is true for fiction and popular nonfiction, but not always required for specialized nonfiction. Some publishers accept non-agented manuscripts. My suggestion is to seek an agent and a publisher simultaneously. To find out how to do this, spend a few hours in a library with Jeff Herman's Guide to Book Publishers, Editors and Literary Agents. Read the articles about how to write query letters, book proposals, select an agent/publisher, etc.
Here are a couple of publishing clichés that became clichés because they are often true:
Fiction is published based on the author’s platform, and published nonfiction is based on the author's credentials.
Nonfiction is sold with a book proposal; novels are sold as a complete manuscript.
This means you should stress your math credentials in your query letter and book proposal. Book proposal formats vary, but they all include a sample chapter, Table of Contents, a section on the author, and a section on the target market.
Don’t worry about a publisher stealing your concepts. Also, if the agent you query is listed in Herman’s book, you don’t need to be concerned about him or her stealing your ideas either. You will need to use your judgment with friends and colleagues.
All of this means you should not wait until your book is complete to your satisfaction. Hone one chapter until it is as good as you can make it and include the other sections required in a book proposal. Then send query letters out to publishers and agents simultaneously. Don’t send a proposal or manuscript unless you get a positive response from a query because it will just end up in a slush pile destined to be read by an intern … someday … perhaps. If you use this approach, you will have plenty of time to complete the entire book to your satisfaction. In fact, publishers assume nonfiction books are not complete at the time of contract signing. A standard clause is a book delivery schedule.
Which brings us to terms and conditions. The sad truth is that unless you are famous or have committed a high-profile felony, you have little influence over the T&Cs, which include royalties. This is true if you negotiate the contract yourself or have an agent negotiate it on your behalf. These contracts are boilerplate for the most part. The agent’s job is to secure the biggest advance possible. My agent also negotiated out a first-rights clause for a second book, but he was able to get little else. Ancillary rights are demanded by traditional publishers. Wiley even insisted on the theatrical rights to my computer management book. (I was thinking of a musical.)
The primary benefit of an agent is to get your manuscript moved to the top of the pile. Agents also know the interests of different publishers and can keep you out of cul-de-sacs. If you query publishers directly, use Herman’s book to select publishers that specialize in your subject or market.
Nowadays, traditional publishers are paying higher royalties on e-books, but nowhere near the direct payments to independent authors. Traditional publishers pay an advance, so they are concerned first with earning back the advance. Indie-authors higher royalties reflect the fact that they receive no advance and pay publication costs.
Traditional publishers will take care of “cleaning up a book.” Wiley assigned an editor and 3 line editors to my book. They also insisted on control over the title and cover. It’s been many years since I published The Digital Organization, and things may have changed, but basically the publisher calls most of the shots.
As for my books, if you are interested in history, I recommend Tempest at Dawn. If you like action/thrillers, then I would recommend The Shopkeeper or The Shut Mouth Society.