A detailed version of the changes with a highly-readable diff is available at the Trac Timeline.
The Mosquito Changelog lists the bug fixes that went into this release. Some of the highlights are:
The following files have changed from version 1.5 to version 1.5.1.
The list was generated against a SVN checkout (April 04, 2005 04:18 UTC) and a 1.5 download using diff -wqrd trunk wordpress
and then modified for readability :
readme.html wp-links-opml.php wp-login.php wp-pass.php wp-rdf.php wp-register.php wp-rss2.php wp-rss.php wp-settings.php xmlrpc.php wp-atom.php wp-blog-header.php wp-comments-post.php wp-commentsrss2.php wp-feed.php wp-admin/admin-functions.php wp-admin/admin-header.php wp-admin/admin.php wp-admin/categories.php wp-admin/edit-form-advanced.php wp-admin/edit-form-comment.php wp-admin/edit-form.php wp-admin/edit-page-form.php wp-admin/edit.php wp-admin/import-blogger.php wp-admin/import-greymatter.php wp-admin/import-mt.php wp-admin/index.php wp-admin/install.php wp-admin/link-add.php wp-admin/link-categories.php wp-admin/link-import.php wp-admin/link-manager.php wp-admin/menu-header.php wp-admin/options.php wp-admin/options-writing.php wp-admin/plugin-editor.php wp-admin/plugins.php wp-admin/post.php wp-admin/profile.php wp-admin/quicktags.js wp-admin/setup-config.php wp-admin/sidebar.php wp-admin/templates.php wp-admin/theme-editor.php wp-admin/themes.php wp-admin/update-links.php wp-admin/upgrade-functions.php wp-admin/upgrade.php wp-admin/upgrade-schema.php wp-admin/user-edit.php wp-admin/users.php wp-admin/wp-admin.css wp-content/plugins/markdown.php wp-content/plugins/textile1.php wp-content/themes/classic/footer.php wp-content/themes/classic/index.php wp-content/themes/default/archive.php wp-content/themes/default/comments.php wp-content/themes/default/comments-popup.php wp-content/themes/default/footer.php wp-content/themes/default/header.php wp-content/themes/default/index.php wp-content/themes/default/search.php wp-content/themes/default/sidebar.php wp-content/themes/default/single.php wp-includes/classes.php wp-includes/class-pop3.php wp-includes/comment-functions.php wp-includes/feed-functions.php wp-includes/functions-compat.php wp-includes/functions-formatting.php wp-includes/functions.php wp-includes/functions-post.php wp-includes/gettext.php wp-includes/links.php wp-includes/streams.php wp-includes/template-functions-author.php wp-includes/template-functions-category.php wp-includes/template-functions-general.php wp-includes/template-functions-links.php wp-includes/template-functions-post.php wp-includes/vars.php wp-includes/version.php wp-includes/wp-l10n.php
/wp-includes/pluggable-functions.php - Contains replaceable functions that can be defined by plugins before being defined in the core.
The Mosquito Bug Tracker has a list of all existing issues. Select the 1.5.1 version for details regarding existing 1.5.1 issues.