基于百度地图SDK和Elasticsearch GEO查询的地理围栏分析系统(2)-查询实现

小说:农业推广项目申请表作者:顺建更新时间:2019-03-19字数:25375

基于百度地图SDK和Elasticsearch GEO查询的地理围栏分析系统(2)-查询实现


 

在上一篇博客中,我们准备好了数据。现在数据已经以我们需要的格式,存放在Elasticsearch中了。

本文讲述如何在Elasticsearch中进行空间GEO查询和聚合查询,以及如何准备ajax接口。

平台的服务端部分使用的springboot+mybatis的基本开发模式。工程结构如下。

可以看到本工程有三个module:

1)moonlight-web是controller和service层的实现;

2)moonlight-dsl封装了ES空间索引查询和聚合查询的方法;

3)moonlight-dao封装了持久化地理围栏的方法。

我们以客户端请求的处理顺序为例进行讲解。

 

1、controller

在controller层中,我们实现了4个接口,分别是circle、box、polygon、heatmap,也就是圆形圈选,矩形圈选,多边形圈选和热力图。

先看一下代码的具体实现。

@RestController
@RequestMapping("/moonlight")
public class MoonlightController {

    protected final Logger logger = LoggerFactory.getLogger(this.getClass());

    @Autowired
    private MoonlightService moonlightService;

    @RequestMapping(value = "/circle", method = RequestMethod.GET)
    public ResponseEntity<Response> circle(HttpServletRequest request, HttpServletResponse response) {
        String point = request.getParameter("point");
        String radius = request.getParameter("radius");
        try {
            Map<String, Object> result = moonlightService.circle(point, radius);
            logger.info("circle圈选成功, points={}, radius={}, result={}", point, radius, result);
            return new ResponseEntity<>(
                    new Response(ResultCode.SUCCESS, "circle圈选成功", result),
                    HttpStatus.OK);
        } catch (Exception e) {
            logger.error("circle圈选失败, points={}, radius={}, result={}", point, radius, null, e);
            return new ResponseEntity<>(
                    new Response(ResultCode.EXCEPTION, "circle圈选失败", null),
                    HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }

    @RequestMapping(value = "/box", method = RequestMethod.GET)
    public ResponseEntity<Response> box(HttpServletRequest request, HttpServletResponse response) {
        String point1 = request.getParameter("point1");
        String point2 = request.getParameter("point2");
        String point3 = request.getParameter("point3");
        String point4 = request.getParameter("point4");
        try {
            Map<String, Object> result = moonlightService.boundingBox(point1, point2, point3, point4);
            logger.info("box圈选成功, point1={}, point2={}, point3={}, point4={}, result={}", point1, point2, point3, point4, result);
            return new ResponseEntity<>(
                    new Response(ResultCode.SUCCESS, "box圈选成功", result),
                    HttpStatus.OK);
        } catch (Exception e) {
            logger.error("box圈选失败, point1={}, point2={}, point3={}, point4={}, result={}", point1, point2, point3, point4, null, e);
            return new ResponseEntity<>(
                    new Response(ResultCode.EXCEPTION, "box圈选失败", null),
                    HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }

    @RequestMapping(value = "/polygon", method = RequestMethod.GET)
    public ResponseEntity<Response> polygon(HttpServletRequest request, HttpServletResponse response) {
        List<String> points = new ArrayList<>();
        Enumeration<String> paramNames = request.getParameterNames();
        while (paramNames.hasMoreElements()) {
            String paramName = paramNames.nextElement();
            if (paramName.startsWith("point")) {
                points.add(request.getParameter(paramName));
            }
        }
        try {
            Map<String, Object> result = moonlightService.polygon(points);
            logger.info("polygon圈选成功, points={}, result={}", points, result);
            return new ResponseEntity<>(
                    new Response(ResultCode.SUCCESS, "polygon圈选成功", result),
                    HttpStatus.OK);
        } catch (Exception e) {
            logger.error("polygon圈选失败, points={}, result={}", points, null, e);
            return new ResponseEntity<>(
                    new Response(ResultCode.EXCEPTION, "polygon圈选失败", null),
                    HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }

    @RequestMapping(value = "/heatMap", method = RequestMethod.GET)
    public ResponseEntity<Response> heatMap(HttpServletRequest request, HttpServletResponse response) {
        try {
            List<Map<String, Object>> result = moonlightService.heatMap();
            logger.info("heatMap请求成功, result={}", result);
            return new ResponseEntity<>(
                    new Response(ResultCode.SUCCESS, "heatMap请求成功", result),
                    HttpStatus.OK);
        } catch (Exception e) {
            logger.error("heatMap请求失败, result={}", null, e);
            return new ResponseEntity<>(
                    new Response(ResultCode.EXCEPTION, "heatMap请求失败", null),
                    HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }
}

我们以圆形圈选(circle接口)为例,circle接口传入两个参数,一个是point,也就是中心点坐标,一个是radius,也就是半径,它干的事情就是圈选出,point点周围radius长度内的所有订单数据,具体实现是调用了service层的方法,controller得到圈选的数据后就返回了。

下面我们来看一下service层。

 

2、service

service层是具体业务的实现。我们这里的service仍然比较简单,可以看到只是初始化了esDao的句柄,然后进行es的geo查询。

先看一下具体代码。

@Service
public class MoonlightService {

    protected final Logger logger = LoggerFactory.getLogger(this.getClass());

    @Autowired
    private ESDao esDao;

    public Map<String, Object> circle(String point, String radius) {
        POI center = new POI(point);
        return esDao.circle(center, Double.parseDouble(radius));
    }

    public Map<String, Object> boundingBox(String point1, String point2, String point3, String point4) {
        POI poi1 = new POI(point1);
        POI poi2 = new POI(point2);
        POI poi3 = new POI(point3);
        POI poi4 = new POI(point4);
        POI topLeft = getTopLeft(poi1, poi2, poi3, poi4);
        POI bottomRight = getBottomRight(poi1, poi2, poi3, poi4);
        logger.info("topLeft - lat={}, lng={}, bottomRight - lat={}, lng={}",
                topLeft.getLat(), topLeft.getLng(), bottomRight.getLat(), bottomRight.getLng());
        return esDao.boundingBox(topLeft, bottomRight);
    }

    public Map<String, Object> polygon(List<String> points) {
        List<POI> poiList = new ArrayList<>();
        for (String point : points) {
            POI poi = new POI(point);
            poiList.add(poi);
        }
        return esDao.polygon(poiList);
    }

    public List<Map<String, Object>> heatMap() {
        return esDao.heatMap();
    }

    private POI getTopLeft(POI poi1, POI poi2, POI poi3, POI poi4) {
        POI topLeft = new POI();
        List<Double> latList = new ArrayList<>();
        List<Double> lngList = new ArrayList<>();
        latList.add(poi1.getLat());
        latList.add(poi2.getLat());
        latList.add(poi3.getLat());
        latList.add(poi4.getLat());
        Collections.sort(latList);
        Double minLat = latList.get(0);
        Double maxLat = latList.get(3);

        lngList.add(poi1.getLng());
        lngList.add(poi2.getLng());
        lngList.add(poi3.getLng());
        lngList.add(poi4.getLng());
        Collections.sort(lngList);
        Double minLng = lngList.get(0);
        Double maxLng = lngList.get(3);

        topLeft.setLat(maxLat);
        topLeft.setLng(minLng);
        return topLeft;
    }

    private POI getBottomRight(POI poi1, POI poi2, POI poi3, POI poi4) {
        POI bottomRight = new POI();
        List<Double> latList = new ArrayList<>();
        List<Double> lngList = new ArrayList<>();
        latList.add(poi1.getLat());
        latList.add(poi2.getLat());
        latList.add(poi3.getLat());
        latList.add(poi4.getLat());
        Collections.sort(latList);
        Double minLat = latList.get(0);
        Double maxLat = latList.get(3);

        lngList.add(poi1.getLng());
        lngList.add(poi2.getLng());
        lngList.add(poi3.getLng());
        lngList.add(poi4.getLng());
        Collections.sort(lngList);
        Double minLng = lngList.get(0);
        Double maxLng = lngList.get(3);

        bottomRight.setLat(minLat);
        bottomRight.setLng(maxLng);
        return bottomRight;
    }
}

我们仍然是以圆形圈选为例,可以看到,service代码的逻辑就是,创建出圈选需要的数据接口,然后调用Dao层进行查询就是了。

circle圈选需要的是一个中心点POI类型,和一个Double半径。

box矩形查询需要的是左上坐标点和右下坐标点,里面有两个函数getTopLeft、getBottomRight分别可以求出矩形的左上点和右下点。

polygon多边形查询需要的是一系列点,这些点顺序的连接所绘制出来的图形就是目标多边形。

heatmap热力图什么参数也不要,将返回一定精度的经纬度计数值,后面我们会详述。

之后所有的service都调用了Dao层的es查询逻辑。所以最重要的一部分是esDao的实现,下面我们就来看一看。

 

3、Dao

Dao层代码是整个项目的核心,包括对Elasticsearch数据进行圈选和聚合两部分,此外就是热力图数据的准备。

先看一下代码。

@Component
public class ESDao {
    protected final Logger logger = LoggerFactory.getLogger(this.getClass());

    @Autowired
    private ESClient esClient;

    public Map<String, Object> circle(POI center, Double radius) {

        TermsQueryBuilder termsQuery = termsQuery("product_id", new double[]{3, 4});

        GeoDistanceRangeQueryBuilder geoDistanceRangeQuery = QueryBuilders.geoDistanceRangeQuery("location")
                .point(center.getLat(),  center.getLng())
                .from("0m")
                .to(String.format("%fm", radius))
                .includeLower(true)
                .includeUpper(true)
                .optimizeBbox("memory")
                .geoDistance(GeoDistance.SLOPPY_ARC);

        QueryBuilder queryBuilder = QueryBuilders.boolQuery().must(termsQuery).must(geoDistanceRangeQuery);

        SearchRequestBuilder search = esClient.getClient().prepareSearch("moon").setTypes("bj")
                .setSearchType(SearchType.DFS_QUERY_AND_FETCH)
                .setQuery(queryBuilder);

        return agg(search);
    }

    public Map<String, Object> boundingBox(POI topLeft,  POI bottomRight) {

        TermsQueryBuilder termsQuery = termsQuery("product_id", new double[]{3, 4});

        GeoBoundingBoxQueryBuilder geoBoundingBoxQuery = QueryBuilders.geoBoundingBoxQuery("location")
                .topLeft(topLeft.getLat(), topLeft.getLng())
                .bottomRight(bottomRight.getLat(), bottomRight.getLng());

        QueryBuilder queryBuilder = QueryBuilders.boolQuery().must(termsQuery).must(geoBoundingBoxQuery);

        SearchRequestBuilder search = esClient.getClient().prepareSearch("moon").setTypes("bj")
                .setSearchType(SearchType.DFS_QUERY_AND_FETCH)
                .setQuery(queryBuilder);

        return agg(search);
    }

    public Map<String, Object> polygon(List<POI> poiList) {

        TermsQueryBuilder termsQuery = termsQuery("product_id", new double[]{3, 4});

        GeoPolygonQueryBuilder geoPolygonQuery = QueryBuilders.geoPolygonQuery("location");
        for (POI poi : poiList) {
            geoPolygonQuery.addPoint(poi.getLat(), poi.getLng());
        }

        QueryBuilder queryBuilder = QueryBuilders.boolQuery().must(termsQuery).must(geoPolygonQuery);

        SearchRequestBuilder search = esClient.getClient().prepareSearch("moon").setTypes("bj")
                .setSearchType(SearchType.DFS_QUERY_AND_FETCH)
                .setQuery(queryBuilder);

        return agg(search);
    }

    public List<Map<String, Object>> heatMap() {

        TermQueryBuilder queryBuilder = termQuery("date", "2017-11-24");
        SearchRequestBuilder searchRequestBuilder = esClient.getClient()
                .prepareSearch("moon").setTypes("bj");
        SearchResponse response = searchRequestBuilder
                .setQuery(queryBuilder)
                .setFrom(0).setSize(10000)
                .setExplain(true).execute().actionGet();

        SearchHits hits = response.getHits();
        Map<String, Integer> countMap = new HashMap<>();
        for (SearchHit hit : hits) {
            Map<String, Object> source = hit.getSource();
            Map<String, Double> locationMap = (Map<String, Double>) source.get("location");
            DecimalFormat df = new DecimalFormat("#.000");
            String lat = df.format(locationMap.get("lat"));
            String lon = df.format(locationMap.get("lon"));
            String key = lat+"-"+lon;
            if (countMap.containsKey(key)) {
                countMap.put(key, countMap.get(key) + 1);
            } else {
                countMap.put(key, 1);
            }
        }
        List<Map<String, Object>> result = new ArrayList<>();
        for (Map.Entry<String, Integer> entry : countMap.entrySet()) {
            String lat = entry.getKey().split("-")[0];
            String lon = entry.getKey().split("-")[1];
            Integer count = entry.getValue();
            Map<String, Object> map = new HashMap<>();
            map.put("lat", Double.parseDouble(lat));
            map.put("lng", Double.parseDouble(lon));
            map.put("count", count);
            result.add(map);
        }
        return result;
    }

    private Map<String, Object> agg(SearchRequestBuilder search) {

        Map<String, Object> resultMap = new HashMap<>();

        GroupBy groupBy = new GroupBy(search, "date_group", "date", true);
        groupBy.addSumAgg("pre_total_fee_sum", "pre_total_fee");
        groupBy.addCountAgg("order_id_count", "order_id");
        groupBy.addSumAgg("cancel_count", "type");

        List<String> xAxis = new ArrayList<>();
        List<String> profits = new ArrayList<>();
        List<String> totals = new ArrayList<>();
        List<String> cancelRatios = new ArrayList<>();
        List<Map<String, Object>> details = new ArrayList<>();

        Map<String, Object> groupbyResponse = groupBy.getGroupbyResponse();
        for (Map.Entry<String, Object> entry : groupbyResponse.entrySet()) {
            String date = entry.getKey();
            xAxis.add(date);
            Map<String, String> subAggMap = (Map<String, String>) entry.getValue();
            String profit = subAggMap.get("pre_total_fee_sum");
            profits.add(profit);
            String total = subAggMap.get("order_id_count");
            totals.add(total);
            String cancelRatioDouble = new DecimalFormat("#.0000").format(
                    Double.parseDouble(subAggMap.get("cancel_count")) / Double.parseDouble(subAggMap.get("order_id_count"))
            );
            String cancelRatio = new DecimalFormat("0.00%").format(
                    Double.parseDouble(subAggMap.get("cancel_count")) / Double.parseDouble(subAggMap.get("order_id_count"))
            );
            cancelRatios.add(cancelRatioDouble);

            Map<String, Object> tempMap = new HashMap<>();
            tempMap.put("profit", profit);
            tempMap.put("total", total);
            tempMap.put("cancelRatio", cancelRatio);
            tempMap.put("date", date);
            details.add(tempMap);
        }

        resultMap.put("xAxis", xAxis);
        resultMap.put("profit", profits);
        resultMap.put("total", totals);
        resultMap.put("cancelRatio", cancelRatios);
        resultMap.put("detail", details);

        return resultMap;
    }

}

es圈选部分

circle为例,我们构造了一个geoDistanceRangeQuery查询,这个查询到上一篇博客准备好的moon索引,bj type中去将数据圈选出来。

类似的我们有矩形geoBoundingBoxQuery查询,多边形geoPolygonQuery查询,具体构造查询的方式可以参照代码,这个代码还是很简单的,熟悉es的同学很快可以上手并且实现这样的查询,不熟悉的话可以自行百度一下。如果还有其他的查询条件,可以通过QueryBuilders.boolQuery().must(termsQuery).must(geoDistanceRangeQuery)加入,例如我这里在圈选之外加入了一个terms查询,这个查询相当于sql中的where product_id in (3,4) and ...。

 

es聚合部分

es聚合部分做的事情是,对查询出的订单进行了聚合运算,例如求和和计数,是两个最常见的运算,这部分在这里不详细叙述了,请参见这篇博客。

 

热力图

这里要额外说明的是,热力图heatmap,和圈选不一样,他是查询了最近一天type=bj分区里的所有数据,按照坐标进行了计数,可以看到的是,计数的时候,我们指定了精度,这里是小数点后三位有效数字

            DecimalFormat df = new DecimalFormat("#.000");
            String lat = df.format(locationMap.get("lat"));
            String lon = df.format(locationMap.get("lon"));
            String key = lat+"-"+lon;

然后将计数结果返回。百度地图SDK会将计数结果绘制成热力图,这个不用我们管,我会在另一篇博客中讲述这个过程。

 

到这里,整个工程的基本功能就介绍完了。

 

当前文章:http://adsl66.com/news/20190266080.html

发布时间:2019-03-19 06:55:14

小程序28分成骗局 极客赚 巨财易赚金官网 赚赚多怎么接不到单 贷款怎么赚钱 小投资网络赚钱平台 好卷佣金怎么提现 怎样找高佣金商品 零花钱的使用手抄报 赚客帝国下载

69973 75243 70656 97770 81929 5399148461 62885 56078

我要说两句: (0人参与)

发布