Negotiators, who spent a week in the talks in the Kenyan capital, have until the end of next year to conclude a treaty to curb the use of plastic, which generates about 400 million tonnes of waste a year.
Environmental group Greenpeace said a successful treaty would require the United States and the European Union to show more leadership than they are showing now.
Graham Forbes, head of the Greenpeace delegation, said: "The harsh truth is that the talks failed to achieve their primary objective, which was to issue a mandate to prepare a preliminary draft of a treaty text."
Two more rounds of talks are due next year in a bid to finalize the treaty.
Participants said that a proposal to hold an additional session before the next round in Canada was not successful in the final plenary.
Bethany Carney-Almerut, an environmental toxicologist at the University of Gothenburg in Sweden, who took part in the talks, said the world was facing a major challenge.
"Plastic is linked to climate change, biodiversity loss and other major threats and crises that we humans face on this planet."
Recycling less than 10%
Stuart Harris, a spokesman for the International Council of the Chemical Association, an industry body that advocates measures such as reusing plastic containers rather than imposing production restrictions, said the Nairobi talks offered good ideas.
A proposal by Switzerland and Uruguay for further discussions on the reduction of harmful polymers and chemicals of concern received the support of more than 100 Member States.
But some participants were disappointed by what they described as the lack of a clear path to an effective agreement.
The U.N. Environment Programme says less than 10 percent of plastic waste is recycled, while the International Union for Conservation of Nature says at least 14 million tonnes end up in the oceans each year.