Preface this with having never gone to a convention but I hope to eventually:
If I were to start a convention I would imagine that the cost of the venue for a weekend would be the first hurdle. This could vary widely depending on the location. Then probably whatever it costs to get a big name or two signed on. Then the money to buy a domain and create the website. Then probably the time or money to hire someone to take the time to market the event to companies to have booths. After that I would think support costs like security, food, tables, personnel to set up and tear down. Then the money to have pamphlets, tickets, and other media made. Then marketing costs to the public.
No doubt it is no small undertaking but the cost of each of these items will be different depending on the location you want to have your con. Altogether it's probably around $1-$2 Million up front to get a decent sized show going.