The Yellow Kid* is considered to be the first comic strip, specifically the October 25, 1896 comic** since it was the only one that matched the specific definition***.
* Mickey Dugan, the Yellow Kid, first appeared in 1894 in Hogan's Alley, then moved with Outcault to another paper where he appeared in MacFadden's Row of Flats. The comic specifically called The Yellow Kid was more of a daily b&w comic featuring you know who.
** Oddly enough no other The Yellow Kid comic ever matched the definition of comic strip.
*** It is specific, something to the effect of a series of sequential pictures with dialogue in word balloons and the words & pictures must work together for the joke to work.
Just think of how many comics we think of as comic strips don't match that definition?
Prince Valiant, nope, it has captions under the pictures.
Henry or The Little King, nope, they don't have word balloons.
Cathy, nope, the humor relies on the wall-of-text, not the art.
The American Dennis the Menace, nope, it's a gag panel, not a series of sequential panels.
As for the first comic book, well...
The oldest known use of the term was in a 1902 advertisement for a book collection of Yellow Kid cartoons.
What we think of as a comic book is usually the floppy magazines sold on newsstands, or stores, and the earliest known newsstand comic seems to be 1933's Detective Dan Secret Agent 48. This however didn't spark other publishers into trying the same thing.
Max Gaines' attempt to sell repackaged comic strips in a floppy magazine for sale at newsstands, & what not, did however encourage other publishers to try to make money the same way.
So depending on how you want to define what a comic book is there is a couple of different answers to what was the first comic book.