java实现自动识别验证码并自动填写提交（调用百度通用文字识别OCR接口+大图找小图之图像识别算法+模拟鼠标键盘动作）

2024-06-27 00:35| 来源: 网络整理| 查看: 265

一、使用的技术：

1.调用百度AI通用文字识别OCR接口

2.图像识别算法

二、实现方案与步骤：

1.得到验证码的图片的坐标：先将验证码截图，然后再对整个电脑屏幕截图，通过大图找小图的方法，计算出小图在大图中的坐标，从而得到验证码的坐标。

2.通过验证码的坐标实现每次截验证码的图片的精准坐标，每次通过该坐标来截图，调用文字识别接口，实现验证码的识别，识别之后，复制到剪贴板。

3.截验证码提交表单的小图，再截整个电脑屏幕，通过大图找小图的方法，计算出小图在大图中的图标，从而得到提交框的坐标。

4.通过java模拟鼠标动作移动到验证码提交框的坐标上再模拟点击鼠标，鼠标的光标在提交框里，让提交框是输入状态，再通过模拟键盘动作Ctrl+v，把刚才复制到剪贴板的验证码粘贴到提交框。

5.截验证码提交按钮的小图，再截整个电脑屏幕，通过大图找小图的方法，计算出小图在大图中的图标，从而得到提交按钮的坐标。

6.通过java模拟鼠标动作移动到验证码提交按钮的坐标上再模拟点击鼠标。

以上就是java实现自动识别验证码并自动填写提交的完整方案。

三、具体实现： 1.如何调用百度AI通用文字识别接口？

（1）在浏览器输入网址http://ai.baidu.com/或者百度搜索‘百度AI’点击官网那个。点击主页的产品服务，看到通用文字识别，选择技术文档，下载Java SDK，解压后得到4个jar包。或者直接从我云盘下：百度网盘提取码：p8sc 把解压后的文件复制粘贴到eclipse的项目的文件夹里。我项目名字是Test1。在这里插入图片描述

（2）刷新项目，选中刚才复制过来的文件夹，右键单击add build path，或者右键项目Build Path -> Add JARs选择那4个jar包，确定。

（3）使用百度ai图片文字识别的接口前还需要 API Key和clientSecret ，得申请一个。首先进入控制台：（前提是已有百度账号：如百度云盘、贴吧等等）在这里插入图片描述选择：产品服务–>人工智能–>文字识别点击：创建应用创建好了就会如下图所示，AppId、API Key、Secret Key都是有值的。到时候在代码里把这些值填进去就好。

在这里插入图片描述

2.代码实现：（1）如何调用这个接口呢？

在我们的项目Test1文件夹里新建java类文件，话不多说，直接上代码：

1.先给一下maven 依赖三个：在这里插入图片描述 2.引用的库： 3.AuthService.java

package test1; import org.json.JSONObject; import java.io.BufferedReader; import java.io.InputStreamReader; import java.net.HttpURLConnection; import java.net.URL; import java.util.List; import java.util.Map; public class AuthService { /** * 获取权限token * @return 返回示例： * { * "access_token": "24.460da4889caad24cccdb1fea17221975.2592000.1491995545.282335-1234567", * "expires_in": 2592000 * } */ public static String getAuth() { // 官网获取的 API Key 更新为你注册的 String clientId = "这里填你申请到的API Key"; // 官网获取的 Secret Key 更新为你注册的 String clientSecret = "这里填你申请到的Secret Key"; return getAuth(clientId, clientSecret); } /** * 获取API访问token * 该token有一定的有效期，需要自行管理，当失效时需重新获取. * @param ak - 百度云官网获取的 API Key * @param sk - 百度云官网获取的 Securet Key * @return assess_token 示例： * "24.460da4889caad24cccdb1fea17221975.2592000.1491995545.282335-1234567" */ private static String getAuth(String ak, String sk) { // 获取token地址 String authHost = "https://aip.baidubce.com/oauth/2.0/token?"; String getAccessTokenUrl = authHost // 1. grant_type为固定参数 + "grant_type=client_credentials" // 2. 官网获取的 API Key + "&client_id=" + ak // 3. 官网获取的 Secret Key + "&client_secret=" + sk; try { URL realUrl = new URL(getAccessTokenUrl); // 打开和URL之间的连接 HttpURLConnection connection = (HttpURLConnection) realUrl.openConnection(); connection.setRequestMethod("GET"); connection.connect(); // 获取所有响应头字段 Map map = connection.getHeaderFields(); // 遍历所有的响应头字段 for (String key : map.keySet()) { // System.out.println(key + "--->" + map.get(key)); } // 定义 BufferedReader输入流来读取URL的响应 BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream())); StringBuilder result = new StringBuilder(); String line; while ((line = in.readLine()) != null) { result.append(line); } /** * 返回结果示例 */ // System.out.println("result:" + result); JSONObject jsonObject = new JSONObject(result.toString()); return jsonObject.getString("access_token"); } catch (Exception e) { //System.out.printf("获取token失败！"); e.printStackTrace(System.err); } return null; } public void main(String[] args) { getAuth(); } }

4.BaseImg64.java

package test1; import sun.misc.BASE64Encoder; import java.io.FileInputStream; import java.io.IOException; import java.io.InputStream; import java.io.UnsupportedEncodingException; import java.net.URLEncoder; /** * 图片转化base64后再UrlEncode结果 */ @SuppressWarnings("restriction") public class BaseImg64 { /** * 将一张本地图片转化成Base64字符串 * @param imgPath 本地图片地址 * @return 图片转化base64后再UrlEncode结果 */ public static String getImageStrFromPath(String imgPath) { InputStream in; byte[] data = null; // 读取图片字节数组 try { in = new FileInputStream(imgPath); data = new byte[in.available()]; in.read(data); in.close(); } catch (IOException e) { e.printStackTrace(); } // 对字节数组Base64编码 BASE64Encoder encoder = new BASE64Encoder(); // 返回Base64编码过再URLEncode的字节数组字符串 try { return URLEncoder.encode(encoder.encode(data),"UTF-8"); } catch (UnsupportedEncodingException e) { // TODO Auto-generated catch block e.printStackTrace(); } return null; } } EasySpecProvider.java package test1; import org.apache.http.cookie.CookieSpec; import org.apache.http.cookie.CookieSpecProvider; import org.apache.http.protocol.HttpContext; public class EasySpecProvider implements CookieSpecProvider { @Override public CookieSpec create(HttpContext arg0) { // TODO Auto-generated method stub return null; } }

6.Check.java 测试看看有没有成功

首先，得在D:\data\image\wjc.png这个路径里，也就是image文件夹里放一张验证码的图片，随便网上下一张验证码的图片即可，不要那种肉眼难辨的文字哦！

import java.net.URISyntaxException; import java.io.File; import java.net.URI; import org.apache.http.client.HttpClient; import org.apache.http.client.config.CookieSpecs; import org.apache.http.client.config.RequestConfig; import org.apache.http.clienthods.CloseableHttpResponse; import org.apache.http.clienthods.HttpGet; import org.apache.http.clienthods.HttpPost; import org.apache.http.entity.StringEntity; import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.DefaultHttpClient; import org.apache.http.impl.client.HttpClients; import org.apache.http.util.EntityUtils; /** * 测试类 * @author Administrator * */ @SuppressWarnings("deprecation") public class Check { private static final String POST_URL = "https://aip.baidubce.com/rest/2.0/ocr/v1/general_basic?access_token=" + AuthService.getAuth(); /** * 识别本地图片的文字 * * @param path 本地图片地址 * @return 识别结果，为json格式 * @throws URISyntaxException URI打开异常 * @throws IOException io流异常 */ public static String checkFile(String path) throws URISyntaxException, IOException { File file = new File(path); if (!file.exists()) { throw new NullPointerException("图片不存在"); } String image = BaseImg64.getImageStrFromPath(path); String param = "image=" + image; return post(param); } /** * @param url 图片url * @return 识别结果，为json格式 */ public static String checkUrl(String url) throws IOException, URISyntaxException { String param = "url=" + url; return post(param); } /** * 通过传递参数：url和image进行文字识别 * * @param param 区分是url还是image识别 * @return 识别结果 * @throws URISyntaxException URI打开异常 * @throws IOException IO流异常 */ private static String post(String param) throws URISyntaxException, IOException { //开始搭建post请求 URI url = new URI(POST_URL); RequestConfig globalConfig = RequestConfig.custom().setCookieSpec(CookieSpecs.IGNORE_COOKIES).build(); CloseableHttpClient client = HttpClients.custom().setDefaultRequestConfig(globalConfig).build(); HttpGet request = new HttpGet(url); HttpClient httpClient = new DefaultHttpClient(); HttpPost post = new HttpPost(); post.setURI(url); //设置请求头，请求头必须为application/x-www-form-urlencoded，因为是传递一个很长的字符串，不能分段发送 post.setHeader("Content-Type", "application/x-www-form-urlencoded"); StringEntity entity = new StringEntity(param); post.setEntity(entity); CloseableHttpResponse response = client.execute(post); // HttpResponse response = httpClient.execute(post); // System.out.println(response.toString()); if (response.getStatusLine().getStatusCode() == 200) { String str; try { /*读取服务器返回过来的json字符串数据*/ str = EntityUtils.toString(response.getEntity()); String []result=str.split(":"); result1 = result[4].split("\""); // System.out.println(result1[1]);//验证码结果 System.out.println(str);//打印得到的数据 return str; } catch (Exception e) { e.printStackTrace(); return null; } } return null; } public static void main(String[] args) throws Exception { String path1 = "D:\\data\\image\\wjc.png";//绝对路径 try { long now = System.currentTimeMillis(); checkFile(path1); //这是测网页上的图片 // checkUrl("https://images0.cnblogs.com/blog/508489/201505/101311124074020.png"); System.out.println("耗时：" + (System.currentTimeMillis() - now) / 1000 + "s"); } catch (URISyntaxException | IOException e) { e.printStackTrace(); } } }

到此，文字识别就实现啦！

2.如何实现大图找小图？

直接上代码！请新建项目测试哦！ CoordBean.java

public class CoordBean { private int x; private int y; public int getX() { return this.x; } public void setX(int x) { this.x = x; } public int getY() { return this.y; } public void setY(int y) { this.y = y; } }

ImageCognition.java

import java.awt.Color; import java.awt.image.BufferedImage; import java.util.ArrayList; import java.util.List; /** * 图像识别 * @author Administrator * */ public class ImageCognition {//精确度 public static final int SIM_ACCURATE_VERY = 0; public static final int SIM_ACCURATE = 31; public static final int SIM_BLUR = 61; public static final int SIM_BLUR_VERY = 81; public List imageSearch(BufferedImage sourceImage, BufferedImage searchImage, int sim) { List list = new ArrayList(); RgbImageComparerBean pxSource = getPX(sourceImage); RgbImageComparerBean pxSearch = getPX(searchImage); int[][] px = pxSource.getColorArray(); int[][] pxS = pxSearch.getColorArray(); int pxSXMax = pxSearch.getImgWidth() - 1; int pxSYMax = pxSearch.getImgHeight() - 1; int xSearchEnd = pxSource.getImgWidth() - pxSearch.getImgWidth(); int ySearchEnd = pxSource.getImgHeight() - pxSearch.getImgHeight(); int contentSearchX = 1; int contentSearchY = 1; double pxPercent = 0.9900000095367432D; if (sim > 0) { pxPercent = sim / 255.0D / 4.0D; } for (int x = 0; x boolean contrast = false; if (sim int pxX = x + pxSearch.getImgWidth() - 1; if (colorCompare(px[pxX][y], pxS[pxSXMax][0], sim)) { int pxY = y + pxSearch.getImgHeight() - 1; if (colorCompare(px[x][pxY], pxS[0][pxSYMax], sim)) { if (colorCompare(px[pxX][pxY], pxS[pxSXMax][pxSYMax], sim)) { if (pxSXMax > 2) { contentSearchX = (int)Math.ceil(pxSXMax / 2); } if (pxSYMax > 2) { contentSearchY = (int)Math.ceil(pxSYMax / 2); } if (colorCompare( px[(x + contentSearchX)][ (y + contentSearchY)], pxS[contentSearchX][contentSearchY], sim)) { contrast = true; } } } } } } else { contrast = true; } if (sim int yes = 0; int ySour = y + contentSearchY; for (int i = 0; i yes++; } } if (yes / pxSearch.getImgWidth() > pxPercent) contrast = true; else { contrast = false; } } if (contrast) { int yes = 0; int xSour = x + contentSearchX; for (int i = 0; i yes++; } } if (yes / pxSearch.getImgHeight() > pxPercent) contrast = true; else contrast = false; } } else { contrast = true; } if (contrast) { int yes = 0; for (int xS = 0; xS if (colorCompare(px[(x + xS)][(y + yS)], pxS[xS][yS], sim)) { yes++; } } } if (yes / pxSearch.getPxCount() > pxPercent) { CoordBean coord = new CoordBean(); coord.setX(x); coord.setY(y); list.add(coord); } } } } return list; } public RgbImageComparerBean getPX(BufferedImage bufferedImage) { int width = bufferedImage.getWidth(); int height = bufferedImage.getHeight(); int minx = bufferedImage.getMinX(); int miny = bufferedImage.getMinY(); RgbImageComparerBean rgb = new RgbImageComparerBean(); int[][] colorArray = new int[width][height]; for (int i = minx; i colorArray[i][j] = bufferedImage.getRGB(i, j); } } rgb.setColorArray(colorArray); return rgb; } public boolean colorCompare(int pxSource, int pxSearch, int sim) { if (sim == 0) { return pxSearch == pxSource; } Color sourceRgb = new Color(pxSource); Color searchRgb = new Color(pxSearch); return colorCompare(sourceRgb, searchRgb, sim); } public boolean colorCompare(Color color1, Color color2, int sim) { return (Math.abs(color1.getRed() - color2.getRed()) return this.colorArray; } public void setPxCount(int pxCount) { this.pxCount = pxCount; } public void setColorArray(int[][] colorArray) { this.colorArray = colorArray; this.imgWidth = this.colorArray.length; this.imgHeight = this.colorArray[0].length; this.pxCount = (this.imgWidth * this.imgHeight); } public boolean[][] getIgnorePx() { return this.ignorePx; } public void setIgnorePx(boolean[][] ignorePx) { this.ignorePx = ignorePx; } public int getImgWidth() { return this.imgWidth; } public int getImgHeight() { return this.imgHeight; } public int getPxCount() { return this.pxCount; } }

Image.java

测试方法：在这里插入图片描述在上图的路径分别截屏小图，就是屏幕的某一个位置的小图，名为search.png 再截全屏，名为screen.png，然后运行一下Image .java，就会生成一个result.png啦！

3.那么问题来了，怎么实现识别验证码并自动提交？

把代码整合起来就好啦！

打字有点累…休息下…

（1）先来说说如何模拟鼠标动作吧！ Robot robot = new Robot(); robot.delay(100); robot.mouseMove(x+20, y+20);//鼠标移动到该坐标 robot.delay(100); //鼠标点击 robot.mousePress(InputEvent.BUTTON1_DOWN_MASK); robot.delay(100); //鼠标释放 robot.mouseRelease(InputEvent.BUTTON1_DOWN_MASK); （2）怎么把一个字符串复制到电脑的剪贴板上呢？ /** * 将字符串复制到剪切板。 */ public static void setSysClipboardText(String writeMe) { Clipboard clip = Toolkit.getDefaultToolkit().getSystemClipboard(); Transferable tText = new StringSelection(writeMe); clip.setContents(tText, null); } （3）又怎么模拟键盘动作呢粘贴字符串？ robot.keyPress(KeyEvent.VK_CONTROL); robot.keyPress(KeyEvent.VK_V); robot.delay(10); robot.keyRelease(KeyEvent.VK_CONTROL); robot.keyRelease(KeyEvent.VK_V); robot.keyPress(KeyEvent.VK_ENTER); robot.delay(10); robot.keyRelease(KeyEvent.VK_ENTER); （4）全部的代码整合：

1.留下以下这些类，把刚才的Image .java删掉，它已经没有价值了。在这里插入图片描述 2.修改Check.java

import java.awt.Color; import java.awt.Dimension; import java.awt.Font; import java.awt.Graphics; import java.awt.Rectangle; import java.awt.Robot; import java.awt.Toolkit; import java.awt.datatransfer.Clipboard; import java.awt.datatransfer.StringSelection; import java.awt.datatransfer.Transferable; import java.awt.event.InputEvent; import java.awt.event.KeyEvent; import java.awt.image.BufferedImage; import java.io.File; import java.io.FileInputStream; import java.io.FileOutputStream; import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; import java.net.URI; import java.net.URISyntaxException; import java.util.List; import java.util.Random; import javax.imageio.ImageIO; import org.apache.http.client.HttpClient; import org.apache.http.client.config.CookieSpecs; import org.apache.http.client.config.RequestConfig; import org.apache.http.clienthods.CloseableHttpResponse; import org.apache.http.clienthods.HttpGet; import org.apache.http.clienthods.HttpPost; import org.apache.http.entity.StringEntity; import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.DefaultHttpClient; import org.apache.http.impl.client.HttpClients; import org.apache.http.util.EntityUtils; /** * 测试类 * @author Administrator * */ @SuppressWarnings("deprecation") //https://aip.baidubce.com/rest/2.0/ocr/v1/general_basic public class Check { private static final String POST_URL = "https://aip.baidubce.com/rest/2.0/ocr/v1/general_basic?access_token=" + AuthService.getAuth(); private static int x=0;//横坐标 private static int y=0;//纵坐标 private static String result1[]=new String[1000]; /** * 识别本地图片的文字 * * @param path 本地图片地址 * @return 识别结果，为json格式 * @throws URISyntaxException URI打开异常 * @throws IOException io流异常 */ public static String checkFile(String path) throws URISyntaxException, IOException { File file = new File(path); if (!file.exists()) { throw new NullPointerException("图片不存在"); } String image = BaseImg64.getImageStrFromPath(path); String param = "image=" + image; return post(param); } /** * @param url 图片url * @return 识别结果，为json格式 */ public static String checkUrl(String url) throws IOException, URISyntaxException { String param = "url=" + url; return post(param); } /** * 通过传递参数：url和image进行文字识别 * * @param param 区分是url还是image识别 * @return 识别结果 * @throws URISyntaxException URI打开异常 * @throws IOException IO流异常 */ private static String post(String param) throws URISyntaxException, IOException { //开始搭建post请求 URI url = new URI(POST_URL); RequestConfig globalConfig = RequestConfig.custom().setCookieSpec(CookieSpecs.IGNORE_COOKIES).build(); CloseableHttpClient client = HttpClients.custom().setDefaultRequestConfig(globalConfig).build(); HttpGet request = new HttpGet(url); HttpClient httpClient = new DefaultHttpClient(); HttpPost post = new HttpPost(); post.setURI(url); //设置请求头，请求头必须为application/x-www-form-urlencoded，因为是传递一个很长的字符串，不能分段发送 post.setHeader("Content-Type", "application/x-www-form-urlencoded"); StringEntity entity = new StringEntity(param); post.setEntity(entity); CloseableHttpResponse response = client.execute(post); // HttpResponse response = httpClient.execute(post); // System.out.println(response.toString()); if (response.getStatusLine().getStatusCode() == 200) { String str; try { /*读取服务器返回过来的json字符串数据*/ str = EntityUtils.toString(response.getEntity()); //对读取到的字符串进行切割 String []result=str.split(":"); result1 = result[4].split("\""); // System.out.println(result1[1]);//验证码结果 // System.out.println(str); return str; } catch (Exception e) { e.printStackTrace(); return null; } } return null; } public static void main(String[] args) throws Exception { String path1 = "D:\\data\\image\\wjc.png"; try { long now = System.currentTimeMillis(); /** * 1.在check之前，要先对验证码截图保存，再截全屏对比得到验证码的坐标，从而之后才能不停地对验证码进行截图，我就不做示范了！ * 2.假装验证码的图片是每一次都不一样的，然后去实现自动填写提交。 * 3.找验证码坐标的原理是和找提交框、提交按钮等一样的，模仿即可！！！ */ checkFile(path1); // checkUrl("https://images0.cnblogs.com/blog/508489/201505/101311124074020.png"); System.out.println("耗时：" + (System.currentTimeMillis() - now) / 1000 + "s"); } catch (URISyntaxException | IOException e) { e.printStackTrace(); } if(!findImage4FullScreen(ImageCognition.SIM_ACCURATE_VERY)) { System.out.println("很抱歉...截图里没找到您想要搜索的图片呢！"); }else { Robot robot = new Robot(); Random random = new Random(); int a = 0; robot.delay(100); robot.mouseMove(x+30, y+30);//鼠标移动到该坐标 a = Math.abs(random.nextInt())%100+50; robot.delay(a); //鼠标点击 robot.mousePress(InputEvent.BUTTON1_DOWN_MASK); a = Math.abs(random.nextInt())%50+50; robot.delay(a); robot.mouseRelease(InputEvent.BUTTON1_DOWN_MASK); String res=result1[1].toString(); System.out.println(res); setSysClipboardText(res); robot.keyPress(KeyEvent.VK_CONTROL); robot.keyPress(KeyEvent.VK_V); robot.delay(10); robot.keyRelease(KeyEvent.VK_CONTROL); robot.keyRelease(KeyEvent.VK_V); robot.keyPress(KeyEvent.VK_ENTER); robot.delay(10); robot.keyRelease(KeyEvent.VK_ENTER); } } /** * 将字符串复制到剪切板。 */ public static void setSysClipboardText(String writeMe) { Clipboard clip = Toolkit.getDefaultToolkit().getSystemClipboard(); Transferable tText = new StringSelection(writeMe); clip.setContents(tText, null); } /** * 大图找小图 * @param sim * @return * @throws Exception */ public static Boolean findImage4FullScreen(int sim) throws Exception { Dimension screenSize = Toolkit.getDefaultToolkit().getScreenSize(); int w = (int) screenSize.getWidth();//屏幕宽度 int h=(int)screenSize.getHeight();//屏幕高度 Robot robot = new Robot(); //截屏操作 BufferedImage screenImg = robot.createScreenCapture(new Rectangle(0, 0, w, h)); OutputStream out = new FileOutputStream("D:\\data\\image\\screen.png"); ImageIO.write(screenImg, "png", out);//将截到的BufferedImage写到本地 InputStream in = new FileInputStream("D:\\data\\image\\search.png"); BufferedImage searchImg = ImageIO.read(in);//将要查找的本地图读到BufferedImage //图片识别工具类 ImageCognition ic = new ImageCognition(); List list = ic.imageSearch(screenImg, searchImg, sim); for (CoordBean coordBean : list) { System.out.println("找到图片了,它的坐标是" +"(" +coordBean.getX() + "," + coordBean.getY()+")"); //标注找到的图的位置 Graphics g = screenImg.getGraphics(); g.setColor(Color.BLACK); g.drawRect(coordBean.getX(), coordBean.getY(), searchImg.getWidth(), searchImg.getHeight()); g.setFont(new Font(null, Font.BOLD, 20)); g.drawString("←找到的图片在这里", coordBean.getX() + searchImg.getWidth() + 5, coordBean.getY() + 10 + searchImg.getHeight() / 2); out = new FileOutputStream("D:\\data\\image\\result.png"); ImageIO.write(screenImg, "png", out); x=coordBean.getX() ; y=coordBean.getY(); return true; } return false; } } （5）关于整合代码之后怎么测试？

这个也很好测试，可以在桌面新建个TXT文档，根据我以上的思路，让它自己写东西，很好很好…

有不懂的可以留言。

【本文地址】

公司简介

联系我们